{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolution Neural Networks\n",
    "In this tutorial we will implement a simple Convolutional Neural Network in TensorFlow with two convolutional layers, followed by two fully-connected layers at the end. The network structure is shown in the following figure and has classification accuracy of above 99% on MNIST data.\n",
    "\n",
    "![CNN](imgs/img1.png)\n",
    "\n",
    "*Fig1. CNN structure used for digit recognition*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. imports\n",
    "First, we have to import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Load the MNIST data\n",
    "\n",
    "### 1.1. Data dimension\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "img_h = img_w = 28  # MNIST images are 28x28\n",
    "img_size_flat = img_h * img_w  # 28x28=784, the total number of pixels\n",
    "n_classes = 10  # Number of classes, one class per digit\n",
    "n_channels = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. Helper functions to load the MNIST data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def load_data(mode='train'):\n",
    "    \"\"\"\n",
    "    Function to (download and) load the MNIST data\n",
    "    :param mode: train or test\n",
    "    :return: images and the corresponding labels\n",
    "    \"\"\"\n",
    "    mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)\n",
    "    if mode == 'train':\n",
    "        x_train, y_train, x_valid, y_valid = mnist.train.images, mnist.train.labels, \\\n",
    "                                             mnist.validation.images, mnist.validation.labels\n",
    "        x_train, _ = reformat(x_train, y_train)\n",
    "        x_valid, _ = reformat(x_valid, y_valid)\n",
    "        return x_train, y_train, x_valid, y_valid\n",
    "    elif mode == 'test':\n",
    "        x_test, y_test = mnist.test.images, mnist.test.labels\n",
    "        x_test, _ = reformat(x_test, y_test)\n",
    "    return x_test, y_test\n",
    "\n",
    "\n",
    "def reformat(x, y):\n",
    "    \"\"\"\n",
    "    Reformats the data to the format acceptable for convolutional layers\n",
    "    :param x: input array\n",
    "    :param y: corresponding labels\n",
    "    :return: reshaped input and labels\n",
    "    \"\"\"\n",
    "    img_size, num_ch, num_class = int(np.sqrt(x.shape[-1])), 1, len(np.unique(np.argmax(y, 1)))\n",
    "    dataset = x.reshape((-1, img_size, img_size, num_ch)).astype(np.float32)\n",
    "    labels = (np.arange(num_class) == y[:, None]).astype(np.float32)\n",
    "    return dataset, labels\n",
    "\n",
    "def randomize(x, y):\n",
    "    \"\"\" Randomizes the order of data samples and their corresponding labels\"\"\"\n",
    "    permutation = np.random.permutation(y.shape[0])\n",
    "    shuffled_x = x[permutation, :, :, :]\n",
    "    shuffled_y = y[permutation]\n",
    "    return shuffled_x, shuffled_y\n",
    "\n",
    "def get_next_batch(x, y, start, end):\n",
    "    x_batch = x[start:end]\n",
    "    y_batch = y[start:end]\n",
    "    return x_batch, y_batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3. Load the data and display the sizes\n",
    "Now we can use the defined helper function in \"train\" mode which loads the train and validation images and their corresponding labels. We'll also display their sizes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Validation-set:\t5000\n"
     ]
    }
   ],
   "source": [
    "x_train, y_train, x_valid, y_valid = load_data(mode='train')\n",
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(len(y_train)))\n",
    "print(\"- Validation-set:\\t{}\".format(len(y_valid)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "logs_path = \"./logs\"  # path to the folder that we want to save the logs for Tensorboard\n",
    "lr = 0.001  # The optimization initial learning rate\n",
    "epochs = 10  # Total number of training epochs\n",
    "batch_size = 100  # Training batch size\n",
    "display_freq = 100  # Frequency of displaying the training results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Network configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 1st Convolutional Layer\n",
    "filter_size1 = 5  # Convolution filters are 5 x 5 pixels.\n",
    "num_filters1 = 16  # There are 16 of these filters.\n",
    "stride1 = 1  # The stride of the sliding window\n",
    "\n",
    "# 2nd Convolutional Layer\n",
    "filter_size2 = 5  # Convolution filters are 5 x 5 pixels.\n",
    "num_filters2 = 32  # There are 32 of these filters.\n",
    "stride2 = 1  # The stride of the sliding window\n",
    "\n",
    "# Fully-connected layer.\n",
    "h1 = 128  # Number of neurons in fully-connected layer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Create network helper functions \n",
    "### 4.1. Helper functions for creating new variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# weight and bais wrappers\n",
    "def weight_variable(shape):\n",
    "    \"\"\"\n",
    "    Create a weight variable with appropriate initialization\n",
    "    :param name: weight name\n",
    "    :param shape: weight shape\n",
    "    :return: initialized weight variable\n",
    "    \"\"\"\n",
    "    initer = tf.truncated_normal_initializer(stddev=0.01)\n",
    "    return tf.get_variable('W',\n",
    "                           dtype=tf.float32,\n",
    "                           shape=shape,\n",
    "                           initializer=initer)\n",
    "\n",
    "\n",
    "def bias_variable(shape):\n",
    "    \"\"\"\n",
    "    Create a bias variable with appropriate initialization\n",
    "    :param name: bias variable name\n",
    "    :param shape: bias variable shape\n",
    "    :return: initialized bias variable\n",
    "    \"\"\"\n",
    "    initial = tf.constant(0., shape=shape, dtype=tf.float32)\n",
    "    return tf.get_variable('b',\n",
    "                           dtype=tf.float32,\n",
    "                           initializer=initial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2. Helper-function for creating a new Convolutional Layer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def conv_layer(x, filter_size, num_filters, stride, name):\n",
    "    \"\"\"\n",
    "    Create a 2D convolution layer\n",
    "    :param x: input from previous layer\n",
    "    :param filter_size: size of each filter\n",
    "    :param num_filters: number of filters (or output feature maps)\n",
    "    :param stride: filter stride\n",
    "    :param name: layer name\n",
    "    :return: The output array\n",
    "    \"\"\"\n",
    "    with tf.variable_scope(name):\n",
    "        num_in_channel = x.get_shape().as_list()[-1]\n",
    "        shape = [filter_size, filter_size, num_in_channel, num_filters]\n",
    "        W = weight_variable(shape=shape)\n",
    "        tf.summary.histogram('weight', W)\n",
    "        b = bias_variable(shape=[num_filters])\n",
    "        tf.summary.histogram('bias', b)\n",
    "        layer = tf.nn.conv2d(x, W,\n",
    "                             strides=[1, stride, stride, 1],\n",
    "                             padding=\"SAME\")\n",
    "        layer += b\n",
    "        return tf.nn.relu(layer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3. Helper-function for creating a new Max-pooling Layer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def max_pool(x, ksize, stride, name):\n",
    "    \"\"\"\n",
    "    Create a max pooling layer\n",
    "    :param x: input to max-pooling layer\n",
    "    :param ksize: size of the max-pooling filter\n",
    "    :param stride: stride of the max-pooling filter\n",
    "    :param name: layer name\n",
    "    :return: The output array\n",
    "    \"\"\"\n",
    "    return tf.nn.max_pool(x,\n",
    "                          ksize=[1, ksize, ksize, 1],\n",
    "                          strides=[1, stride, stride, 1],\n",
    "                          padding=\"SAME\",\n",
    "                          name=name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4. Helper-function for flattening a layer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def flatten_layer(layer):\n",
    "    \"\"\"\n",
    "    Flattens the output of the convolutional layer to be fed into fully-connected layer\n",
    "    :param layer: input array\n",
    "    :return: flattened array\n",
    "    \"\"\"\n",
    "    with tf.variable_scope('Flatten_layer'):\n",
    "        layer_shape = layer.get_shape()\n",
    "        num_features = layer_shape[1:4].num_elements()\n",
    "        layer_flat = tf.reshape(layer, [-1, num_features])\n",
    "    return layer_flat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.5. Helper-function for creating a new fully-connected Layer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fc_layer(x, num_units, name, use_relu=True):\n",
    "    \"\"\"\n",
    "    Create a fully-connected layer\n",
    "    :param x: input from previous layer\n",
    "    :param num_units: number of hidden units in the fully-connected layer\n",
    "    :param name: layer name\n",
    "    :param use_relu: boolean to add ReLU non-linearity (or not)\n",
    "    :return: The output array\n",
    "    \"\"\"\n",
    "    with tf.variable_scope(name):\n",
    "        in_dim = x.get_shape()[1]\n",
    "        W = weight_variable(shape=[in_dim, num_units])\n",
    "        tf.summary.histogram('weight', W)\n",
    "        b = bias_variable(shape=[num_units])\n",
    "        tf.summary.histogram('bias', b)\n",
    "        layer = tf.matmul(x, W)\n",
    "        layer += b\n",
    "        if use_relu:\n",
    "            layer = tf.nn.relu(layer)\n",
    "        return layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Network graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1. Placeholders for the inputs (x) and corresponding labels (y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('Input'):\n",
    "    x = tf.placeholder(tf.float32, shape=[None, img_h, img_w, n_channels], name='X')\n",
    "    y = tf.placeholder(tf.float32, shape=[None, n_classes], name='Y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2. Create the network layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "conv1 = conv_layer(x, filter_size1, num_filters1, stride1, name='conv1')\n",
    "pool1 = max_pool(conv1, ksize=2, stride=2, name='pool1')\n",
    "conv2 = conv_layer(pool1, filter_size2, num_filters2, stride2, name='conv2')\n",
    "pool2 = max_pool(conv2, ksize=2, stride=2, name='pool2')\n",
    "layer_flat = flatten_layer(pool2)\n",
    "fc1 = fc_layer(layer_flat, h1, 'FC1', use_relu=True)\n",
    "output_logits = fc_layer(fc1, n_classes, 'OUT', use_relu=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3. Define the loss function, optimizer, accuracy, and predicted class\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with tf.variable_scope('Train'):\n",
    "    with tf.variable_scope('Loss'):\n",
    "        loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=output_logits), name='loss')\n",
    "    tf.summary.scalar('loss', loss)\n",
    "    with tf.variable_scope('Optimizer'):\n",
    "        optimizer = tf.train.AdamOptimizer(learning_rate=lr, name='Adam-op').minimize(loss)\n",
    "    with tf.variable_scope('Accuracy'):\n",
    "        correct_prediction = tf.equal(tf.argmax(output_logits, 1), tf.argmax(y, 1), name='correct_pred')\n",
    "        accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name='accuracy')\n",
    "    tf.summary.scalar('accuracy', accuracy)\n",
    "    with tf.variable_scope('Prediction'):\n",
    "        cls_prediction = tf.argmax(output_logits, axis=1, name='predictions')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.4. Initialize all variables and merge the summaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Initialize the variables\n",
    "init = tf.global_variables_initializer()\n",
    "# Merge all summaries\n",
    "merged = tf.summary.merge_all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training epoch: 1\n",
      "iter   0:\t Loss=2.30,\tTraining Accuracy=25.0%\n",
      "iter 100:\t Loss=0.43,\tTraining Accuracy=85.0%\n",
      "iter 200:\t Loss=0.33,\tTraining Accuracy=92.0%\n",
      "iter 300:\t Loss=0.24,\tTraining Accuracy=93.0%\n",
      "iter 400:\t Loss=0.08,\tTraining Accuracy=98.0%\n",
      "iter 500:\t Loss=0.08,\tTraining Accuracy=99.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 1, validation loss: 0.11, validation accuracy: 97.0%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 2\n",
      "iter   0:\t Loss=0.12,\tTraining Accuracy=98.0%\n",
      "iter 100:\t Loss=0.12,\tTraining Accuracy=96.0%\n",
      "iter 200:\t Loss=0.09,\tTraining Accuracy=98.0%\n",
      "iter 300:\t Loss=0.10,\tTraining Accuracy=98.0%\n",
      "iter 400:\t Loss=0.09,\tTraining Accuracy=97.0%\n",
      "iter 500:\t Loss=0.09,\tTraining Accuracy=98.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 2, validation loss: 0.07, validation accuracy: 97.9%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 3\n",
      "iter   0:\t Loss=0.02,\tTraining Accuracy=99.0%\n",
      "iter 100:\t Loss=0.08,\tTraining Accuracy=97.0%\n",
      "iter 200:\t Loss=0.11,\tTraining Accuracy=96.0%\n",
      "iter 300:\t Loss=0.09,\tTraining Accuracy=97.0%\n",
      "iter 400:\t Loss=0.04,\tTraining Accuracy=98.0%\n",
      "iter 500:\t Loss=0.06,\tTraining Accuracy=98.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 3, validation loss: 0.06, validation accuracy: 98.4%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 4\n",
      "iter   0:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 100:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 200:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 400:\t Loss=0.03,\tTraining Accuracy=98.0%\n",
      "iter 500:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 4, validation loss: 0.06, validation accuracy: 98.2%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 5\n",
      "iter   0:\t Loss=0.02,\tTraining Accuracy=100.0%\n",
      "iter 100:\t Loss=0.04,\tTraining Accuracy=99.0%\n",
      "iter 200:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.13,\tTraining Accuracy=96.0%\n",
      "iter 400:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 500:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 5, validation loss: 0.05, validation accuracy: 98.4%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 6\n",
      "iter   0:\t Loss=0.04,\tTraining Accuracy=99.0%\n",
      "iter 100:\t Loss=0.05,\tTraining Accuracy=98.0%\n",
      "iter 200:\t Loss=0.02,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.02,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.03,\tTraining Accuracy=98.0%\n",
      "iter 500:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 6, validation loss: 0.04, validation accuracy: 98.8%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 7\n",
      "iter   0:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 100:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 200:\t Loss=0.02,\tTraining Accuracy=99.0%\n",
      "iter 300:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.07,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 7, validation loss: 0.05, validation accuracy: 98.8%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 8\n",
      "iter   0:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "iter 100:\t Loss=0.02,\tTraining Accuracy=99.0%\n",
      "iter 200:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 8, validation loss: 0.05, validation accuracy: 98.6%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 9\n",
      "iter   0:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 100:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "iter 200:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.02,\tTraining Accuracy=99.0%\n",
      "iter 500:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 9, validation loss: 0.04, validation accuracy: 98.9%\n",
      "---------------------------------------------------------\n",
      "Training epoch: 10\n",
      "iter   0:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 100:\t Loss=0.03,\tTraining Accuracy=99.0%\n",
      "iter 200:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "iter 300:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 400:\t Loss=0.01,\tTraining Accuracy=100.0%\n",
      "iter 500:\t Loss=0.00,\tTraining Accuracy=100.0%\n",
      "---------------------------------------------------------\n",
      "Epoch: 10, validation loss: 0.04, validation accuracy: 98.8%\n",
      "---------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "sess = tf.InteractiveSession()\n",
    "sess.run(init)\n",
    "global_step = 0\n",
    "summary_writer = tf.summary.FileWriter(logs_path, sess.graph)\n",
    "# Number of training iterations in each epoch\n",
    "num_tr_iter = int(len(y_train) / batch_size)\n",
    "for epoch in range(epochs):\n",
    "    print('Training epoch: {}'.format(epoch + 1))\n",
    "    x_train, y_train = randomize(x_train, y_train)\n",
    "    for iteration in range(num_tr_iter):\n",
    "        global_step += 1\n",
    "        start = iteration * batch_size\n",
    "        end = (iteration + 1) * batch_size\n",
    "        x_batch, y_batch = get_next_batch(x_train, y_train, start, end)\n",
    "\n",
    "        # Run optimization op (backprop)\n",
    "        feed_dict_batch = {x: x_batch, y: y_batch}\n",
    "        sess.run(optimizer, feed_dict=feed_dict_batch)\n",
    "\n",
    "        if iteration % display_freq == 0:\n",
    "            # Calculate and display the batch loss and accuracy\n",
    "            loss_batch, acc_batch, summary_tr = sess.run([loss, accuracy, merged],\n",
    "                                                         feed_dict=feed_dict_batch)\n",
    "            summary_writer.add_summary(summary_tr, global_step)\n",
    "\n",
    "            print(\"iter {0:3d}:\\t Loss={1:.2f},\\tTraining Accuracy={2:.01%}\".\n",
    "                  format(iteration, loss_batch, acc_batch))\n",
    "\n",
    "    # Run validation after every epoch\n",
    "    feed_dict_valid = {x: x_valid, y: y_valid}\n",
    "    loss_valid, acc_valid = sess.run([loss, accuracy], feed_dict=feed_dict_valid)\n",
    "    print('---------------------------------------------------------')\n",
    "    print(\"Epoch: {0}, validation loss: {1:.2f}, validation accuracy: {2:.01%}\".\n",
    "          format(epoch + 1, loss_valid, acc_valid))\n",
    "    print('---------------------------------------------------------')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None, title=None):\n",
    "    \"\"\"\n",
    "    Create figure with 3x3 sub-plots.\n",
    "    :param images: array of images to be plotted, (9, img_h*img_w)\n",
    "    :param cls_true: corresponding true labels (9,)\n",
    "    :param cls_pred: corresponding true labels (9,)\n",
    "    \"\"\"\n",
    "    fig, axes = plt.subplots(3, 3, figsize=(9, 9))\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(np.squeeze(images[i]), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            ax_title = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            ax_title = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        ax.set_title(ax_title)\n",
    "\n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "\n",
    "    if title:\n",
    "        plt.suptitle(title, size=20)\n",
    "    plt.show(block=False)\n",
    "    \n",
    "def plot_example_errors(images, cls_true, cls_pred, title=None):\n",
    "    \"\"\"\n",
    "    Function for plotting examples of images that have been mis-classified\n",
    "    :param images: array of all images, (#imgs, img_h*img_w)\n",
    "    :param cls_true: corresponding true labels, (#imgs,)\n",
    "    :param cls_pred: corresponding predicted labels, (#imgs,)\n",
    "    \"\"\"\n",
    "    # Negate the boolean array.\n",
    "    incorrect = np.logical_not(np.equal(cls_pred, cls_true))\n",
    "\n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    incorrect_images = images[incorrect]\n",
    "\n",
    "    # Get the true and predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "    cls_true = cls_true[incorrect]\n",
    "\n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=incorrect_images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9],\n",
    "                title=title)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "---------------------------------------------------------\n",
      "Test loss: 0.04, test accuracy: 98.9%\n",
      "---------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAggAAAI7CAYAAACJEmNgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcFNXV//HvAUFUFERwA2GMuBFU\nXIhiXHhUDOIa4r4EjUvELTE+ksSgolHEJeqjElGMYkRcwBXi9lODgriB4L4ioEQRWUQlKAj390fV\nJN19apjunl5nPu/Xq19DnbpVfWq403361u0qCyEIAAAgVbNyJwAAACoPBQIAAHAoEAAAgEOBAAAA\nHAoEAADgUCAAAACHAgFAo2dmvc0smNmQcucCVAsKBFQsM9vGzG40s7fMbImZLTezz8zsH2Z2spm1\nKneOxWRmE80s5wuVmNmJ8Zvh6h6zi5AygEZkjXInACQxs4skXayoiH1J0p2SvpW0kaTekm6TNFDS\nLmVKsRq8LunhOtZ9VcpEAFQfCgRUHDO7QNIlkj6VdEQI4eWENgdJOq/UuVWZGSGEIeVOAkB14hQD\nKoqZ1UgaImmFpH5JxYEkhRAmSOqbsP2RZvZ8fEpimZm9aWZ/NLM1E9rOjh/rmdm18b9X1J6nNrMh\n8XB8bzM71sxeNrNvM4fnzWxXMxtnZvPi0yCfmtktZrZpHcfYzswuj0+d/DvO9XUzG2Zm65hZTXxq\nYe+4feqpgYlZ/zKzZGa/i/f9QMK6/cxsZfx7XCslfpiZjTazD8xsafx7mWZm55iZe10xs1Hxc2xu\nZmeZ2Ttm9l38O7/AzCxud4SZvRLvc76Z3ZR0Kqn2d2Fmm5rZXXHbZXEOx+Z4/O3M7AozezfexxIz\ne8bM9k9o2zI+xtfMbHH8/zfbzB4xs/1yeV6g0jGCgEpzkqQWku4NIby1uoYhhO9Tl81sqKQ/Slog\naYyiUxIHSBoq6Wdm1ieEsCJjNy0lPSupnaSnJH0taVZGm/Mk9ZE0XtI/JbVJec6TJI2U9L2kRxWN\nemwp6RRJB5vZbiGET1Labx7vo4ukaZJuVlSobyXpXEkjFA3/XyLpxLjdJSm5zF7d7yQfIYRrzay3\npP5mdkYI4a9xrhtLGi3pO0lHhRCWpWw2TNIqSS9L+pei38k+kv5PUk9JJ9TxdNcoOkU0XtHv+xBJ\nl0tqaWaL4v0+LGmSot/5mZKaKzqdlGl9SVMU/b7ukNRW0pGS7jazjiGEq+s7djPrImmipJr4OZ+Q\ntI6kgyQ9YWa/DiGMTNlklKRjJL0l6e+SlknaVNIeigrWp+t7TqBqhBB48KiYh6RnJAVJp+S4Xa94\nu08kbZwSX0PRm1GQdEHGNrPj+NOS1knY55B4/VJJOyas30rSckkfSeqYsW4fSSslPZQRfyHe5x8T\n9tdeUquU5YnRn2jOv8MT4+eYER9D0qNvxjYbKCpulknaQVHR8nS8n5MSnmOLhFgzRXNFgqRdM9aN\niuOzU39Xit7UF8S/4y8lbZuybk1J7ygqvjbM2F+IH/dLapYS31zSovj/5Ucp8d5x+yEZ+5moqNA5\nOiPeNv79LZO0URxrE7edKql5wvFvUO6/Hx48CvkoewI8eKQ+4jeEkPkGlsV2I+PtTktYt1X8Zv1x\nRry2QNihjn3WFgjX1bH+unj9gXWsf0jSD5LWjZd3jttPT31TW80xNbRAWN3j+oTt9ojzfU/RqEuQ\nNDrH594p3u6ijHhtgXBywja3x+suTVh3cbxu74x4iHPdfDX/bxenxFyBoKgQCpLG1nEsh8brz4iX\n14uXX5Bkpf7b4MGj1A9OMaDSWPwz16/37RT/fDZzRQjhAzObK2lzM2sbQkidwf+dpDfq2fcrdcR7\nxT/3NrOeCes3VDQ8vpWi0wm7xfEnQwir6nnOQrgzhHBito1DCJPN7GJJlyk6VfOhpNOT2prZBpLO\nl9RP0o8UDcun6ljH00xNiH0W/5yWsO5f8c9OCes+CSFkng6SosLqYkk71pFDrdr/vzaWfH2EDvHP\nbSUphPC1mY2XdLCkGfGcjUmSXg4h/Lue5wKqDgUCKs1nkrZR8hvC6tTOC/i8jvWfS+oct0stEOaH\nEOorRubVEd8g/nl+Pdu3jn+2jX/+q66GFeBBSZcqOl1wWwjh28wGZtZW0quKhvNfUXQufpGiT/Rt\nJf1G0emBJEsSYj9ksa5Fwrov6niO2v+vNnWsr1X7/9cnftSldcq/j5L0e0nH6r9zQ74zs3GS/jeE\nUFdOQNXhWwyoNJPjn/vmuF3tm8vGdazfJKNdrWxGKupqU7uvNiEEW83jubhdbWFS16frsoq/LXBP\nvLhY0kVmtnVC01MUFQeXhBB2DSGcEUIYHKKvVN5XmmwlRdfESFLbB5IKjlS1639Tz//fSbUbhBCW\nhRCGhBC2UlRwHq+ozx4vaVwDjgWoOBQIqDR3KPqK4y/MrNvqGmZ8dXF6/LN3QruuikYkZmWcXmio\nl+Kfe+bY/mdJXwVMsFKSzKx5ronl6VpF5+WvkHS0pLUl3ZfwNcOu8U/3tUjFX80skc4WfS02U+/4\n5/SEdaly/f9LE0L4NIRwt6SfKTods0d86gVoFCgQUFFCCLMVTTJrKekfZpZ4pUQz6yvp8ZTQ7fHP\nwWbWIaVdc0VfrWsm6W8FTvcmRcXMdWa2VUKOLc3sP28+IYRpir6W10PRMHVm+w0y3owXxj87FzTr\nBGb2C0VfJXxB0eS+pyRdpahguDaj+ez4Z++MfeyoaO5CqTSXdGVqsRV/jfQcRacmRq9u4xDCVEVz\nCPqb2a+S2pjZdma2YfzvDma2a0KzdSStGz/n8nwOBKhEzEFAxQkhDDWzNRRNNHvVzKYomtxWe6nl\nvRRda2BqyjZTzOwqSYMkvRWfE16q6DoI3RUNA9f7vfgc83wvfmO5XdLbZvaEpA8UnS/vrOiT6ZeK\n5lTUOl7RJLqh8ZvyREUTM7eUtH/cdnbc9hlJR0h60MweU/SVuzkhhLuyTLFHHZPvavMfIv3n4lS3\nKTqtcGwIYWXcZLCi3/VAM3smhFA7YvB3RfMurjez/1H06XlLRdcOeFDRefpSeEPSrpKmmdlTiuYc\nHKVoHsSgEMLMLPZxrKKJrX8zs3MUXdfhK0UjTtsr6ju9JM1XdGroJTN7V9Jrir4Wup6i495Y0g0h\nhG8Kd3hAmZX7axQ8eNT1UDR7/EZFF6X5WtGns88VjRycLGnNhG2OVlQMfKPoGwpvS/qTUq4vkNJ2\ntqTZq3n+IYrmH/SuJ8/tFH2Nb46i7+wvinO+RdI+Ce03kHSlpPfjHL9S9J37yyWtndKuuaKvG36s\naKQiSJqYxe/tRNX/NccQt22haKg9SOqfsK8uigqHr5TylUJJ3RRdGGq+okJsmqK5CTXxvkZl7GdU\nHK/J5fecciwnZsSDouJqU0UjBfPj3+VrioqczP30VsJ1EOJ160q6ID6GbxUVYrMk/UPSaYqvkaGo\n8LhIUUHxr/j/+vM4j2PEVx95NLKHhZDrt8kAoLziS1E/F0LoXe5cgMaKOQgAAMChQAAAAA4FAgAA\ncJiDAAAAHEYQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIAAHAoEAAAgEOB\nAAAAHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAA\nhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIVcTMRpvZ\nkHLnARQKfRqNUWPp12UvEMzs25THKjNblrJ8XBnyeSojp+VmNj3LbU8xs5Xxdl+b2XQz61fsnOvI\n5cKM41gW57Z+OfJpSiqwT//BzN42s2/M7GMz+10O21ZSnz7EzKaY2Vdm9rmZ3WJmrcuRS1NUgf16\nXzObGPfLj3LctpL6dUczGx/36WBmncqRR5KyFwghhNa1D0mfSDo4JXZ3ZnszW6PI+eyfkdMrksbm\nsItJ8XbrS/q7pLFm1iazUQmO488Zx/EXSc+EEBYX83lReX06dryktpIOlHSumR2ew7YV0aclrSvp\nEkmbSPqxpM0lDSvycyJWgf16qaTbJP0+z+0rpV+vkvSYpFz+Jkui7AVCfczsMjO7z8zuMbNvJB2f\nOXxjZvuZ2eyU5U5m9pCZfWlms8zszDyfewtJvSTdleu2IYSVkm6XtLakzWtzNLMLzGyepJHxcxxi\nZq/Hn4omm1n3lOff2cxmxJ/87pG0Zp7HYZJOkHRnPtujsErdp0MIw0II00MIK0MI70oaL+mnueZd\n7j4dQrg7hPBkCGFZCGGRojeHnI8DxVGGfv1SCGG0pFkNybsC+vXnIYSbJU1ryHEUQ8UXCLGfSxoj\nqY2k+1bX0MyaS5og6VVJHSX1kXS+me0br9/bzBZk+bwDJP0zhPBprgnHVefJkr6RNDMOd5LUWlJn\nSWeYWU9Fne8USRso6qSPmFlLM1tT0iNxrF3878NSjzPuqLtlkc7/KKqSH8r1OFA0ZenTZtZM0h6S\n3s414Qrr05K0Vz7HgaIq12t13iqwX1eMaikQJocQxocQVoUQltXTdjdJ64UQhoYQlocQPpL0N0lH\nS1II4bkQQvv6njDlU/eoHHPdw8y+kjRP0ZDRYSGEb+J1P0gaEue1TNJpkv4aQng1/nR3e9yup6JP\nRkHSjSGEFSGEeyX9Zy5E3L5tCOGlLHIaIOn+EMK/czwWFE/J+3Tsz4r64d9zyLXi+rSZHSDpWEkX\n53AcKL5y9et8VFy/rjSlOPdZCLl8gu8iqXP8H1+ruaSJOT7n3ooqxQdz3G5yCKF3Heu+CCEsT1nu\nIuk4Mzs3JdZSUTXdUtLcEEJIWTcnx1xkZutI+oWkA3LdFkVV8j5tZr9R9OK7Z0Y/rE+l9endFZ32\n6x9CmFlfe5RUOV6r81VR/boSVUuBEDKWlyo6X1Rr45R/fyrpwxDCtg18zgGSxhX4U3fmcXwq6ZIQ\nwpWZDeNhtszZrJ2V+5Dq4ZK+kDQ5x+1QXCXt02Z2mqTzJO0VQvgs3/0kKGmfNrNdJD0s6ZchhIm5\npYoSKMdrdTGU47W64lTLKYZMMyQdaGbrm9kmks5JWfeipOVmdp6ZtYrP/2xnZjtnu3MzW1vRG+uo\nhHWTzWxwA/OvdaukM82sp0Vam9nB8af+yZKamdlZZraGmR0haac8nmOApDszqltUnqL1aTMboGj2\nf58QwuyE9VXRp81sB0Wzvc8IITxWoHxRXMXs183MrJWkFtGitTKzFinrq6Jfx7m20n8nNq4Zz2so\nu2otEEZJelfRMM4Tku6tXRFC+EFSP0k/kTRb0gJJt0haT5LMrHfGkFaSX0j6UtKkhHWdJL3QoOz/\nm+vLkgZKulnSYkkfKPo6mkII3yua8HNqvK6/ok9Okv4z8eVbM+tV1/7NrLOiiVw5fwsDJTdKxevT\nlyk6XTbN/vu99ZtS1ldLn/7f+DhGpRzH64XIG0UzSsXr1/tIWibpUUk/iv/9eMr6qujX8STJZZJq\nj/UjRSMvZWd8sMyemdVIuiuEsGeZUwEKgj6Nxoh+XRgUCAAAwKnWUwwAAKCIKBAAAIBDgQAAAJyc\nroPQvn37UFNTU6RUUO2mTZu2IITQodx55II+jdWhT6OxyaVP51Qg1NTUaOrUqfllhUbPzKru6mH0\naawOfRqNTS59mlMMAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgUCAAA\nwKFAAAAADgUCAABwKBAAAIBDgQAAAJyc7uYIoPJcc801acvLli1zbd544w0XGzduXFb7HzhwoIv1\n6tXLxU444YSs9gegOjCCAAAAHAoEAADgUCAAAACHOQhAFTnqqKNcbOzYsXnty8yyajdixAgXe/rp\np11s7733Tlvu3LlzXnkB5fDBBx+42NZbb522fMMNN7g2Z599dtFyKjdGEAAAgEOBAAAAHAoEAADg\nUCAAAACHSYpAhSrkhMRtttnGxfr27etiH3/8sYs9+uijLvbRRx+52OjRo9OWL7jgglxSBMpq+vTp\nLtasWfpn6I4dO5YqnYrACAIAAHAoEAAAgEOBAAAAHAoEAADgMEkRqABTp051sYceeiirbbt37562\nnDSpsH379i7WunVrF1u+fLmL7brrri72+uuvu9jChQtXmydQyWbMmOFimX8j/fv3L1U6FYERBAAA\n4FAgAAAAhwIBAAA4FAgAAMCp+EmK48aNc7GRI0e62KabbupirVq1crHjjjsubXnjjTd2bbp27ZpL\nikCDff755y4WQnCxzAmJkvTkk0+mLW+yySZ553HNNde42LvvvpvVtgcddFDezwuU0ptvvuliN954\no4v98pe/LEU6FYsRBAAA4FAgAAAAhwIBAAA4FAgAAMCp+EmK559/vovNnj077/2NGDEibXm99dZz\nbbp165b3/otts802c7FBgwa52C677FKKdFAgBx98sIsl3VJ53XXXdbF27doVLI/77rvPxZKurghU\ns/fff9/Fli5d6mJJt1xvShhBAAAADgUCAABwKBAAAIBDgQAAAJyKn6R42223uVjSrWaTJha+8847\nLjZ9+vS05YkTJ7o2L730kot17tzZxT755BMXy1aLFi3SlpNux5t0db2k3JImLjJJsfp16dKlqPu/\n+uqrXeyDDz7IatukW0AnxYBKdNVVV7lYTU2NizX111FGEAAAgEOBAAAAHAoEAADgUCAAAACn4icp\n7rvvvlnFkvTt27feNosXL3axzImMUvJklVdffTWrPJKsueaaactbb721a7PNNtu42KJFi1xsiy22\nyDsPNA0TJkxwsYsuusjFvv/+exfbaKONXGzYsGEutvbaa+eZHVA8SVfeTXrtTnoNXmeddYqRUtVg\nBAEAADgUCAAAwKFAAAAATsXPQSi29ddf38X22WefrLbNdi5ENh544AEXS5ofsf3227vY0UcfXbA8\n0DhNnTrVxZLmGyRJuqPd3nvv3eCcgFJ47rnnsmrXoUOHImdSfRhBAAAADgUCAABwKBAAAIBDgQAA\nAJwmP0mxXObPn5+2fMYZZ7g2IQQXS7q4Tbt27QqXGKreYYcd5mJPPvlkVtsOGDDAxS677LIG5wSU\nyxtvvJFVu0GDBhU5k+rDCAIAAHAoEAAAgEOBAAAAHAoEAADgMEmxTIYPH562nDlpUZLatm3rYkl3\nHEPT9vnnn6ctT5kyxbVJumpi0pXjBg8e7GKtW7duQHZA6bz44osudscdd7jYjjvu6GJ9+vQpSk7V\njBEEAADgUCAAAACHAgEAADgUCAAAwGGSYglMnjzZxYYNG1bvdo888oiLde/evSA5ofHo379/2vKC\nBQuy2u64445zsS222KIgOQHl8Mwzz7jY4sWLXaxv374u1qpVq6LkVM0YQQAAAA4FAgAAcCgQAACA\nQ4EAAAAcJimWwGOPPeZiy5cvT1veb7/9XJtevXoVLSdUp0cffdTFpk+fXu92vXv3drFLL720ECkB\nFeP111/Pqt0RRxxR5EwaB0YQAACAQ4EAAAAcCgQAAOBQIAAAAIdJigW2bNkyF3viiSdcbM0110xb\nvuSSS1ybFi1aFC4xVJ2FCxe62NChQ10sc8Jrkh49ergYt3FGNZs3b56LTZo0ycW22WYbF/v5z39e\nlJwaG0YQAACAQ4EAAAAcCgQAAOBQIAAAAIdJigV29dVXu1jSle4OOOCAtOXdd9+9aDmhOv3lL39x\nsVdeeaXe7Q477DAX46qJaGxGjRrlYl988YWLZb7WInuMIAAAAIcCAQAAOBQIAADAYQ5CA0yYMMHF\n/vznP7tYmzZtXOzCCy8sSk5oPK699tq8ths+fLiLcVEkNDZz5szJqt36669f5EwaL0YQAACAQ4EA\nAAAcCgQAAOBQIAAAAIdJillKurPeOeec42I//PCDi/Xr18/FevXqVZjEgAxJfbXQdwZNmnib+Rwr\nVqxwbZYsWZLV/hcvXuxi1113XZbZpWvevLmLXXnllS629tpr57V/lMf48eOzanfQQQcVOZPGixEE\nAADgUCAAAACHAgEAADgUCAAAwGGSYoKVK1e6WN++fV1s1qxZLta1a1cXS7q6IlAs22+/fdGf48gj\nj3SxTTbZJG056c569957b9FyysVGG23kYoMHDy5DJsjGpEmTXCypf6GwGEEAAAAOBQIAAHAoEAAA\ngEOBAAAAHCYpJpg5c6aLTZ06Nattk27Ru8UWWzQ4JzQ9SVfgfPjhh8uQiXf//fcXbF9JV3ls1iy7\nzy6HHHKIi+2yyy71brfHHntktX9UhoceesjFkq5au+OOO7rY3nvvXZScmgJGEAAAgEOBAAAAHAoE\nAADgUCAAAACnyU9SnDNnjovtv//+WW17zTXXuBi3FkWhPPjggy521VVXudjy5cvz2v8777zjYg25\n0uHJJ5+cttylS5estvvFL37hYttuu23eeaD6/fvf/05bfvzxx7Pa7ogjjnCxpNt9IzuMIAAAAIcC\nAQAAOBQIAADAoUAAAABOk5+keMstt7hY0sTFJElX6DKzBucE1GXQoEFF3f+YMWOKun8gG5lX12zb\ntq1rc+ihh7rYb37zm6Ll1BQxggAAABwKBAAA4FAgAAAAhwIBAAA4TW6S4qRJk9KWb7rppjJlAgBI\nkjlJ8cUXXyxTJk0bIwgAAMChQAAAAA4FAgAAcJrcHITJkyenLX/zzTdZbde1a1cXa926dUFyAgCg\n0jCCAAAAHAoEAADgUCAAAACHAgEAADhNbpJiNnr06OFizzzzjIu1a9euFOkAAFByjCAAAACHAgEA\nADgUCAAAwKFAAAAATpObpPjHP/5xtcsAAIARBAAAkIACAQAAOBQIAADAoUAAAACOhRCyb2z2paQ5\nxUsHVa5LCKFDuZPIBX0a9aBPo7HJuk/nVCAAAICmgVMMAADAoUAAAAAOBQIAAHAoEAAAgEOBAAAA\nHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIB\nAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIVcTMJpvZieXO\nAygk+jUaGzMbbWZDyp1HQ5W9QDCzb1Meq8xsWcrycWXIp5mZXWNmi8xsoZldYWaW5baXmdmKOPev\nzOwFM9u12DmvJp/zzWyemS0xs9vMrGW5cmlqKq1fp+S1ppl9aGazc9imovp1Sl7PmVkodx5NRaX1\naTPb18wmmtnXZvZRjtueYmYr49y/NrPpZtavWLnWk0tHMxtvZp+bWTCzTuXII0nZC4QQQuvah6RP\nJB2cErs7s72ZrVHklAZK6iepu6QekvpLOjmH7e+Oj2VDSS9LeiCpUbGPw8wOlHSepP+RtLmkrSVd\nVMznxH9VYL+u9QdJn+exXUX065TnGSApq8IdhVGBfXqppNsk/T7P7SfFx7K+pL9LGmtmbTIbleA4\nVkl6TNLhRX6enJW9QKhP/OnlPjO7x8y+kXR85vCNme2X+onIzDqZ2UNm9qWZzTKzM3N4ygGSrgkh\nfBZC+FTStZJOzDXvEMJySXdK6mhmbeOK9Xkzu8HMFkkaHOd6ipm9Z2aLzexxM9ss5Tj6mtn78QjA\n/ym3F8QBkm4NIbwbQlgk6bJ8jgPFUYZ+LTPrKukoSVflm3cF9GuZ2fqS/qSo2EGFKHWfDiG8FEIY\nLWlWQ/IOIayUdLuktSVtXpujmV1gZvMkjYxzPcTMXo9H0SabWfeU49jZzGaY2Tdmdo+kNXN4/s9D\nCDdLmtaQ4yiGii8QYj+XNEZSG0n3ra6hmTWXNEHSq5I6Suoj6Xwz2zdev7eZLVjNLn4s6fWU5dfj\nWE7MbE1Fb8izQwhfxeHdJb0rqYOkK83scEnnSzo0jr2s6DhlZhtKGqfoRbC9pLmSdk3Z/+ZxR900\nh+PomFQho2xK2a8l6SZFn7a+yzfhCujXkjRM0o2S5ud7HCiaUvfpBotHCE6W9I2kmXG4k6TWkjpL\nOsPMeioqFE6RtIGiguIRM2sZ/008Esfaxf8+LPU44z69W7GPpdCqpUCYHEIYH0JYFUJYVk/b3SSt\nF0IYGkJYHkL4SNLfJB0tSSGE50II7ZM2NDNTVEUuSQkvkbRuDrkea2ZfSfpU0naK/mBqfRJCuDmE\nsDI+jl9LGhpCeD+E8IOiT/k/MbOOkg6SNCOE8FAIYYWkv0j6snZHIYRZIYS2IYTP6sijdcJxKMdj\nQXGVpF9LkpkdIemHEML4PHOtiH5t0dyHnpL+mudxoLhK1qcLYI+4T89TNLx/WAjhm3jdD5KGxHkt\nk3SapL+GEF6N+/ntcbuekn4qKUi6MYSwIoRwr6TptU8St28bQnipiMdSFKU679lQn+bQtoukzvF/\nfK3mkibWt2EIIZjZvyWtlxJeT1Flma0xIYQT61iXeRxdJA2Ph1lrrVJUvW6a2j6EsMrM5uaQx7fy\nx1EbR2UoSb82s9aSrpD0s5yyS1f2fm1mzRQVBmeHEFZadnOHUVol6dMFMjmE0LuOdV/Ep9NqdZF0\nnJmdmxJrqWjko6WkuSGE1AmzcwqaaZlUS4GQOVN5qaJP+rU2Tvn3p5I+DCFsm+dzvS1pB0mvxcs7\nxLFCyDyOTyVdGEJwQ3Hx+a2+KcvNFL3AZqv2OB6Ml3eQ9K+UYWGUX6n69TaKhkpfiN9UW0pqE59f\n7RnPtWmIUvXrdoomDj8QH0fzeB/zJPUPIUzJPXUUWClfq4spqU9fEkK4MrNhfEoksw93VuHeN8qm\nWk4xZJoh6UAzW9/MNpF0Tsq6FyUtN7PzzKxVfP5nOzPbOct9/13SeWa2qUVfNzlX0qjalWY218yO\nL9BxjJD0JzPbNt532/j8rRSdm+thZofG58jOVXQ+N1t/l3SqmW1jZu0UTR4bVaC8URzF6tczFL1g\n9Ygfv5b0Wfzvz6Sq6dcLFX1iqz2Og+N4D0lTC5Q7Cqtor9UWfSW9laQW0aK1MrMWKesnm9ngAh3H\nrZLONLOeFmltZgeb2TqSJktqZmZnmdka8em8nXLZeXwctRMb14znNZRdtRYIoxRNipoj6QlJ99au\niM959pP0E0mzJS2QdIviIXaawT+fAAAfVklEQVQz650xpJXpr5KeVFT9vaFowsnf4m1bKfpKzMuF\nOIgQwlhF35IYa2Zfx8/3s3jdF4pmnF+t6IWxc+rzmtmPLPoOb+JkrhDCBEnXSXpe0e/hQ0mXFiJv\nFM0oFaFfhxB+CCHMq31IWixpZby8slr6dYikHseCOD4vYzgYlWOUivdavY+kZZIelfSj+N+Pp6zv\nJOmFQhxECOFlRV+Bv1nR388Hko6P132vaE7OqfG6/pIert02Lny+NbNeSfuOC+VlkmqP9SNFIy9l\nZ+mnTbA6ZtZb0skhhBPKnQtQKPRrNDZmViPprhDCnmVOpapRIAAAAKdaTzEAAIAiokAAAAAOBQIA\nAHByug5C+/btQ01NTZFSQbWbNm3aghBCLl/FLDv6NFaHPo3GJpc+nVOBUFNTo6lT+boxkplZ1V09\njD6N1aFPo7HJpU9zigEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIB\nAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOGuUO4FKtHTpUhc7\n//zzXWzEiBEutssuu7jY2LFjXaxLly55ZgcAQPExggAAABwKBAAA4FAgAAAAhwIBAAA4TFJM8Nln\nn7nYyJEjXax58+YuNnXqVBcbP368i5111ll5Zgeke+2119KW+/fv79rMnj27RNms3lNPPeVi2267\nrYttttlmpUgH+I+k1+lDDjkkbfnGG290bQYOHOhiSe8N1YgRBAAA4FAgAAAAhwIBAAA4FAgAAMBp\n8pMUv/zySxcbMGBAGTIB8vPkk0+mLX///fdlyqR+jz76qIvdfvvtLnbvvfeWIh00UQsXLnSxpMmG\nmc4++2wXO/nkk11srbXWyi+xCsMIAgAAcCgQAACAQ4EAAAAcCgQAAOA0uUmKN9xwQ9ryww8/7Nq8\n+uqrBX3OSZMmuVgIIW15hx12cG322muvguaB6vfDDz+42GOPPVaGTPKTdDv0a6+91sWSbrm+zjrr\nFCUnND3PP/+8i/3rX/+qd7tjjjnGxVq1alWQnCoRIwgAAMChQAAAAA4FAgAAcCgQAACA0+QmKf72\nt79NWy7FbTkffPDBemOdO3d2be6//34X23nnnQuXGKrOP//5TxebMmVK2vLvf//7UqWTs0WLFrnY\n22+/7WL//ve/XYxJishH0pVFL7vssrz2dcIJJ7iYmeW1r2rACAIAAHAoEAAAgEOBAAAAHAoEAADg\nNOpJiv369XOxzCsYrly5sqDP2b59exdLmlw1Z86ctOVZs2a5Nj179nSxVatWNSA7VJM333zTxY4+\n+mgX69q1a9ryBRdcULScGirpds9AMb3xxhsu9tprr2W17RprpL9FHnDAAQXJqVowggAAABwKBAAA\n4FAgAAAAhwIBAAA4jWaS4nPPPedi7733notlXvWqIVdSPP30011s//33d7E2bdq42LPPPpu2fPnl\nl2f1nDfffLOLDRw4MKttUV2S+kTSFQZHjx6dtty6deui5ZSLpKsmJv2dNuYr0aH8kq5km60+ffoU\nMJPqwwgCAABwKBAAAIBDgQAAAJyqnIMwe/ZsF0u6gMyCBQvy2n/SnRUPP/xwF7v44otdbO21187q\nObp06ZK2fMstt7g2SfkPGjTIxb777jsXO+uss1ysRYsWWeWG0hs3bpyLPfbYYy6WeVEkKfmCWpUg\n6Y55SfMNevfu7WJt27YtRkpogpLmvSRp2bKliw0dOrTQ6VQVRhAAAIBDgQAAABwKBAAA4FAgAAAA\npyonKa5YscLF8p2QuNdee7nYfffd52JJd2lsiMxJikl34Pvd737nYkuXLnWxpImLhxxyiIttscUW\nuaSIEho7dqyLJf1fV/JFsTInD48ZM8a1ybw7niQNHjzYxZhQi3xMmTLFxV588cWstk2aYN6jR48G\n51TNGEEAAAAOBQIAAHAoEAAAgEOBAAAAnKqcpNgQmVedu+OOO1ybQk9IzEbSpMK7777bxV555ZVS\npIMiWrJkiYu99NJLWW17xhlnFDqdgrn11lvTlr/88kvXplu3bi62zz77FC0nNC2vvvpq3ttW8gTg\ncmEEAQAAOBQIAADAoUAAAAAOBQIAAHAazSTFlStXZtXu5ZdfLnIm+QkhuNiqVauyapd07Em3oh49\nenSe2aGQvv/+exebO3euix1zzDGlSKdgZs6cWW+b7t27lyATNFXZTlJMup14JU8ALhdGEAAAgEOB\nAAAAHAoEAADgUCAAAACnKicpjhgxwsWaN29ehkwKZ/z48S42ffp0FzMzF0s69ksuuaQwiaHg1l13\nXRdLuq3sm2++6WKLFi1ysXbt2hUmsRzMnz/fxZJuWZ3ppz/9aTHSQRM0efJkF0u6xXiSNm3auFin\nTp0anFNjwwgCAABwKBAAAIBDgQAAABwKBAAA4FTlJMUJEyaUO4WcJN329p133klbHjp0aN77T7o9\ndYsWLfLeH4prrbXWcrGuXbu62Lhx41zswAMPdLHf/e53hUlM0ltvveViSVdInDNnjoslTaDN1KwZ\nn0lQGAsXLnSxpCvNJunTp0+h02mU+GsFAAAOBQIAAHAoEAAAgEOBAAAAnKqcpFhtLr/8chcbPnx4\nXvuqqalxsTvvvNPFOnfunNf+UR5DhgxxsaQJV0kTdI8++uiC5dGhQwcXS5p8uGDBgrz2f9JJJ+W1\nHZApmyt3Ssm3dj7ttNMKnU6jxAgCAABwKBAAAIBDgQAAABzmIBRYv379XOy9994r2P67devmYnvu\nuWfB9o/y2HbbbV3s/vvvd7GkO3wmXcgoX4cffnhW7QYMGOBio0ePrne7pItEAfWZO3eui2V758ak\nuzT27NmzwTk1BYwgAAAAhwIBAAA4FAgAAMChQAAAAE5VTlJMuoDMypUrs9r28ccfr7fNqaee6mKf\nffZZVvtPyi2bu9xlq9ruZInC2nHHHbOKFduPfvSjvLZ78803XWy77bZraDpo5KZMmeJi2d658dBD\nDy10Ok0GIwgAAMChQAAAAA4FAgAAcCgQAACAU5WTFAcOHOhigwYNymrbAw88MG25efPmWW2Xbbuk\nyZLZbpvp9NNPz2s7oNiSJohlM2mMCYnIx8KFC7Nq1759exf77W9/W+h0mgxGEAAAgEOBAAAAHAoE\nAADgUCAAAACnKicp9u/f38WuuuoqF1uwYEEp0qlX0sSZzNv7jhw50rXZZJNNipYT0BBJVwct5BVD\ngVRPPvlkVu0222wzF2vTpk2h02kyGEEAAAAOBQIAAHAoEAAAgEOBAAAAnKqcpNilSxcXu++++1zs\n4YcfdrHrr7++KDmtzp/+9CcXO+uss0qeB1Ao3333Xb1t1lprrRJkgsZoxYoVacsfffRRVtu1atXK\nxVq0aFGQnJoiRhAAAIBDgQAAABwKBAAA4FAgAAAApyonKSbZa6+9sortv//+acu33nqrazN+/HgX\nO/jgg13s17/+tYsl3fK2W7duLgZUszvuuMPF2rZtm7Z80UUXlSodNDLNmqV/du3Zs6dr8/bbb7vY\nlltuWbScmiJGEAAAgEOBAAAAHAoEAADgUCAAAACn0UxSzFbfvn1XuwygfkmTxs4999y05X322adU\n6aCRad68edry5Zdf7tok3V58p512KlpOTREjCAAAwKFAAAAADgUCAABwmtwcBAANl3QxMaBYNt10\nUxe7/fbby5BJ08IIAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIA\nAHAoEAAAgEOBAAAAHAoEAADgUCAAAACHAgEAADgWQsi+sdmXkuYULx1UuS4hhA7lTiIX9GnUgz6N\nxibrPp1TgQAAAJoGTjEAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4FAg\nAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQAAOBQIAAAAIcCAQAAOBQIAADA\noUAAAAAOBQIAAHAoEAAAgEOBAAAAHAqEKmJmk83sxHLnARSKmY02syHlzgMopMbyWl32AsHMvk15\nrDKzZSnLx5UxrzXN7EMzm53DNpeZ2Yo496/M7AUz27WIaWab13NmFsqdR1NRaX3azNY3s7vM7Esz\nm29mF+aw7SlmtjLO/Wszm25m/YqZ72pyMTO7yMw+iXMZY2aty5FLU1SB/Tr19bb20TmPbcv6Wp3x\nN1b72LMcuWQqe4EQQmhd+5D0iaSDU2J3Z7Y3szVKlNofJH2ex3Z3x8eyoaSXJT2Q1KhUx2FmAyRZ\nKZ4LkQrs0zdIaiGps6TdJP3KzE7IYftJ8bGsL+nvksaaWZvMRiU4jl9JOlpSL0kdJa0n6f+K/JyI\nVWC/luLX25THJ7luq8p4rZ6UcRyTSvCc9Sp7gVCfuNK7z8zuMbNvJB2fOSxpZvulftI3s05m9lD8\niWmWmZ2Z43N2lXSUpKvyzTuEsFzSnZI6mlnbuEp83sxuMLNFkgbHz3WKmb1nZovN7HEz2ywlj75m\n9r6ZLTGz/1OOb/Rmtr6kPykqdlAhytCnD5J0ZQhhWQjhY0l3KHqzzUkIYaWk2yWtLWnz2hzN7AIz\nmydpZJzrIWb2evzJbLKZdU85jp3NbIaZfWNm90haM4cUDpY0MoTwrxDCN4r+Po8xs1a5HgsKrxyv\n1YVQCa/VlariC4TYzyWNkdRG0n2ra2hmzSVNkPSqok8ZfSSdb2b7xuv3NrMF9TzfTZJ+L+m7fBM2\nszUlnShpdgjhqzi8u6R3JXWQdKWZHS7pfEmHxrGXFR2nzGxDSeMUvbm3lzRX0q4p+988fgHedDVp\nDJN0o6T5+R4HiqaUfdqU/oJlkrrX0XZ1eawh6WRJ30iaGYc7SWqtaHTiDDPrqahQOEXSBooKikfM\nrGX8N/FIHGsX//uw1OOM+/RuORzHWpK2yPVYUDSlfq0+zMwWmdlbZvbrfBKukNfqXcxsQVxk/Cn+\n3ZRdtRQIk0MI40MIq0IIy+ppu5uk9UIIQ0MIy0MIH0n6m6KhSYUQngshtK9rYzM7QtIPIYTxeeZ6\nrJl9JelTSdsp+oOp9UkI4eYQwsr4OH4taWgI4f0Qwg+SLpP0EzPrqOhT34wQwkMhhBWS/iLpy9od\nhRBmhRDahhA+q+M4dpXUU9Jf8zwOFFfJ+rSkJyT9wcxam9mWil4M184h1z3iPj1P0uGSDos/wUvS\nD5KGxHktk3SapL+GEF6N+/ntcbuekn4qKUi6MYSwIoRwr6TptU8St28bQnipjjwel3SamXUxs7aS\nBsXxXI4FxVXKfn2PpG0VvWGfLunS+PU7WxXxWi3pn4oK9g0lHSHpBEm/y+E4iqZU5/Mb6tMc2naR\n1Dn+j6/VXNLE+ja0aMLTFZJ+llN26caEEE6sY13mcXSRNDwekqq1StGnsk1T24cQVpnZ3GwSMLNm\nigqDs0MIK80axWhXY1OSPh07S9FI0kxFL1z3SPpFDs8/OYTQu451X8RDtKm5Hmdm56bEWir6hNhS\n0twQQuqE2Tk55DFS0d/G84o+3FwnqZ+iT2yoDCXr1yGEt1MWJ5vZjYoK2LFZPn/ZX6vj9jNTFt8w\ns8sknS3p6mz3USzVUiBkzsBfqvRPDRun/PtTSR+GELbN43m2UTRU+kL8ptpSUpv4/GrPEEIunT9J\n5nF8KunCEIIbiovP2/ZNWW6mqDNmo52kHpIeiI+jebyPeZL6hxCm5J46CqxUfVohhAWSjqldNrOr\nJL2Sz76Sdp+x/KmkS0IIV2Y2jIeOM/twZ0lvZ7ZNfKJoDsRg/feccL/4+eblmDOKp2T9uo7nLtSn\noVK9Vtf13BXxqa5aTjFkmiHpQIu+vrWJpHNS1r0oabmZnWdmreLzmtuZ2c5Z7rezojfXHoqGlT6L\n//2ZJJnZXDM7vkDHMULSn8xs23jfbeNzXVJ0bq6HmR0an/s9V9FQWjYWKvrEVnscB8fxHpKmFih3\nFFax+rTMrKuZtTOzNczsQEXzCC5PWT/ZzAYX6DhulXSmmfW0SGszO9jM1pE0WVIzMzsrzuUISTtl\nu2Mza29mP4r3213SNYpOb/AV3spVzH59WPyaafEp1bMUzWupXV8Nr9UyswPieQwys26KJpY/svqt\nSqNaC4RRiiaQzFF0fvXe2hXx+aF+kn4iabakBZJuUfSVKJlZ74whLaVuG0KYV/uQtFjSynh5pUWz\npddXNEGlwUIIYyVdq+hrY19LekPx6Y0QwheKvklxtaI3/M6pzxu/UH6bNPElRFKPY0Ecn5cxHIzK\nMUpF6NOxnoo+pX8t6c+SjgohvJeyvpOkFwpxECGElyUNlHSzor+fDyQdH6/7XtF53lPjdf0lPVy7\nbfwG8a2Z9apj9x0U/W6WKnpRviVljgMq0ygVr18fK+ljRZNm75R0We3XLavltTq2v6S3zGyppPGS\n7pfkRuDKwSi+s2dmvSWdHELI5TvkQMUysxpJd4UQKuLCLEAh8FpdGBQIAADAqdZTDAAAoIgoEAAA\ngEOBAAAAnJyug9C+fftQU1NTpFRQ7aZNm7YghJD113sqAX0aq0OfRmOTS5/OqUCoqanR1Kl8jR7J\nzCyXq+JVBPo0Voc+jcYmlz7NKQYAAOBQIAAAAIcCAQAAOBQIAADAoUAAAAAOBQIAAHAoEAAAgEOB\nAAAAHAoEAADgUCAAAACHAgEAADgUCAAAwKFAAAAADgUCAABwKBAAAIBDgQAAABwKBAAA4KxR7gQA\nAKhEixcvTlv+5JNP8t5Xly5dXOy6665zse7du7vYVlttlba8ww475J1HLhhBAAAADgUCAABwKBAA\nAIBDgQAAAJxGPUlx/vz5LnbkkUemLe++++6uzWmnneZiNTU1Bcur0JYsWeJizz//vIv17dvXxVq0\naFGUnACgUk2YMMHFxo8f72ITJ05MW/7www/zfs6tt97axWbPnu1i33//fb37WrVqVd555IIRBAAA\n4FAgAAAAhwIBAAA4FAgAAMBpNJMUM694JUk//vGPXSxzQt9GG23k2lTbhMSddtrJxRYsWOBiU6dO\ndbEtt9yyMImhYnz99dcu9oc//MHF3n77bRd7+umnXYyJrKhEM2fOdLHhw4e72K233upiy5Ytc7EQ\nQmESq8P7779f1P0XAyMIAADAoUAAAAAOBQIAAHAoEAAAgFOVkxSTJuBlXiFRkhYuXOhiZ555Ztry\njTfeWLjESuCyyy5zsVmzZrlY0sQcJiQ2TqNHj05bHjx4sGuT7W1qkyY4brDBBvklBhTR3LlzXez6\n668vQybeNtts42JJt3GudIwgAAAAhwIBAAA4FAgAAMChQAAAAE5VTlJ87bXXXCzztpx1ueiiiwqc\nTfG89dZbLnbNNde42M9//nMXO+qoo4qSE8oraWLWueeem7acNInXzLLa/9lnn+1iN910k4u1a9cu\nq/0BqZL6ZtLEwj322CNtOelW9S1btnSxNm3auFjr1q1d7Ntvv3Wxn/3sZy6WObFw1113dW123HFH\nF1trrbVcbJ111nGxSscIAgAAcCgQAACAQ4EAAAAcCgQAAOBU/CTF+fPnu9gDDzyQ1ba33367i3Xo\n0KHBORVD0oTEPn36ZLVt//79XWzddddtcE6oPEmTVJOuGJqve++918Uef/xxF0u6WmPmBMekSWRo\nOpYuXepiSa9pr7/+uos9/PDD9e6/V69eLjZ9+nQXq6mpcbGkK4t26tTJxZo1a9qfoZv20QMAgEQU\nCAAAwKFAAAAADgUCAABwKn6S4nnnnedimbe3laSddtrJxY444oii5FQMkydPdrF58+a52EknneRi\nxx9/fFFyQnnNmTPHxe644456t9thhx1cbKONNnKx//f//l9WeSxZssTFkiZLHnfccWnLG2+8cVb7\nR/Vbvny5ix177LEuljQh8YILLnCx/fbbL688kiYkJuncuXNe+29qGEEAAAAOBQIAAHAoEAAAgFPx\ncxCS7kKXFOvYsaOLVcqFWpYtW+ZiQ4cOTVsePny4a5N0nEkXf0LjNGPGDBf7+uuvXWyvvfZKW37u\nuedcm++++87FxowZ42JXXHGFi3300UculjQ/5tBDD01bTrrAEneBrH5Jd0LMfD2TpPHjx7tY0oXq\nzj//fBdbe+2188wOhcQIAgAAcCgQAACAQ4EAAAAcCgQAAOBU/CTFbE2YMMHF9t9/fxdr27Zt2vLA\ngQMLmsfEiROzir300kv17quaLvSEwvv+++9dLGni6rnnnlvvvlq1auViv/rVr1xs3LhxLjZz5kwX\nCyG4WObEskqZJIzCSrrT4rBhw1ysS5cuLjZp0iQXa9OmTWESQ8ExggAAABwKBAAA4FAgAAAAhwIB\nAAA4FT9J8Te/+Y2LPfvssy722WefuVjSFeUyJ1c98sgjDcjOS5q8lTSxLNMWW2zhYklXJ0PTcc89\n92TV7h//+Efa8mGHHZb3c06dOjXvbXfbbbe05datW+e9L1SuKVOmZNVuxx13dLFOnToVOh0UESMI\nAADAoUAAAAAOBQIAAHAoEAAAgFPxkxR33nlnF3vzzTddLOnWuE888YSLXXXVVWnLG264oWszYMCA\nXFJMc8IJJ7jY9ttvX+92u+++u4slTVxE03HMMce4WNKk2ldffTVt+b333nNtkv5mHnroIRdbvHix\ni2VefbSudrfeemvactLfQrdu3VwM1SXpaptJkm73fckll7jYIYcc4mJJExxReowgAAAAhwIBAAA4\nFAgAAMChQAAAAI4lXfmvLrvssktoyJXWmoKPP/7YxZImG/bo0SNt+amnnnJtOnToULjESsDMpoUQ\ndil3Hrmo5D69aNEiF0vqS0uWLElbzvdqnpLUp08fFxs+fLiLHXTQQS72wQcfpC2fdtpprs2IESOy\nyqNS0Ke9pL6Ubf9K0rx5cxc7/fTT05Z33XVX1+bTTz91sa5du7rYj3/846zyePvtt12sV69eacuN\n4UqQufRpRhAAAIBDgQAAABwKBAAA4FAgAAAAp+KvpFhtLr30UhdLmsCTeUXHapuQiOJr166di40d\nO9bFDj/88LTlzEmLUvLExXPOOcfFrrzyShdr1aqVi/Xv39/FrrjiirTlJ5980rWZOXOmi3HF0Ory\nv//7vy72l7/8Je/9rVy50sUyJ8YmTZQthcwr7fbu3du1uffee0uUTekxggAAABwKBAAA4FAgAAAA\nhwIBAAA4TFJsgKQJY3feeaeLrbfeei62wQYbFCUnNG777befi2XefnfMmDGuTdItm5Mm1CZNSExy\n4YUXuti7776btpx0a+qk50z6m0HlGjZsmIsdeeSRLnbccce52IoVK1xs7ty5LpY0cbEc5s+fn7ac\n9JrfvXt3Fxs8eHDRciolRhAAAIBDgQAAABwKBAAA4DAHoQEef/zxrNodeOCBLrbTTjsVOh00UZnz\nEpLmKRTaWmut5WJHHXVU2nLSHIR//vOfLpZ018qki0ShMiTdfbFnz54ulnl3z7o888wzLpY5V2HI\nkCGuzSuvvJLV/gsp6YJj06ZNK3kepcIIAgAAcCgQAACAQ4EAAAAcCgQAAOAwSbEBkiYprrPOOi6W\ndPczoLHJvFjOo48+6tok3fnupptucrGLLrqocImhou277771tpkxY4aLJU1SbNGihYuddNJJLnbq\nqae62HXXXediSRcda0oYQQAAAA4FAgAAcCgQAACAQ4EAAAAcJilmacSIES42b948F9too41cjKsm\noilo1iz988agQYNcm4cfftjFkq6Sd/TRR7vYVlttlX9yqGr777+/i11wwQUulnS3yFtvvdXFPvzw\nQxebOHFiXrl17Ngxr+2qASMIAADAoUAAAAAOBQIAAHAoEAAAgMMkxSwlTVI0Mxfr169fVvv75ptv\n0pYXL17s2nTu3DnL7IDK06NHDxf785//7GJJVxr94x//6GKjR492saTbTqPx2XbbbV0s8/biknTf\nffdltb+k244nWWON9LfIAw880LW58sors9pXNWIEAQAAOBQIAADAoUAAAAAOBQIAAHCYpFhgmZNa\npOTJVZm3Fu3evbtrc+eddxYuMaAC/PKXv3SxW265xcUefPBBF0u6+t32229fmMRQ0ZImo15//fUu\nljn5W5KmTZvmYl988YWL1dTUuFhmf0266mdjxggCAABwKBAAAIBDgQAAABwKBAAA4DBJscBGjhzp\nYrfddpuLnXLKKWnLF154YdFyAipFhw4dXOzpp592sS5durjYsGHDXGzMmDGFSQxVZ6ONNnKxCRMm\nuNhdd93lYi+++KKLJU1A3HDDDfNLrpFgBAEAADgUCAAAwKFAAAAADgUCAABwmKSYpRtvvNHFLr74\nYhfba6+9XGzgwIEutv7666ctt2zZsgHZAdUr6bbmffr0cbFHH33Uxd555x0X69atW2ESQ6Nwwgkn\nZBWDxwgCAABwKBAAAIBDgQAAABzmIGRpzz33dLFnn322DJkAjd+4ceNcbIcddnCxjz76yMWYgwAU\nBiMIAADAoUAAAAAOBQIAAHAoEAAAgMMkRQAVZ7311nOxWbNmlSEToOliBAEAADgUCAAAwKFAAAAA\nDgUCAABwKBAAAIBDgQAAABwKBAAA4FAgAAAAhwIBAAA4FkLIvrHZl5LmFC8dVLkuIYQO5U4iF/Rp\n1IM+jcYm6z6dU4EAAACaBk4xAAAAhwIBAAA4FAgAAMChQAAAAA4FAgAAcCgQAACAQ4EAAAAcCgQA\nAOBQIAAAAOf/Az1MPs5st6mbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1f589e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAggAAAI7CAYAAACJEmNgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xe4FPX5///XjQqoiGJQQRDQGGLv\nNVHAGGMLFtToR7HEWGKJPcagRlRETX722FFJwPa1i71r0GgCikZFrDQrGBEElHb//pg5cXfvOZzd\nZffsKc/HdZ0L5t7ZmXvOeZ89977n3hlzdwEAAORqU+sEAABA00OBAAAAAgoEAAAQUCAAAICAAgEA\nAAQUCAAAIKBAQKMys+fMrFE+W2tmE81sYmPsqxRm1s/M3MwGZzy2hZk9aWbT03XGpfHh6XKvKuY1\nON1Hv2rto6Uzs8PS7+Fhtc4FWFIUCCha+sLnZrbIzH64mPWezVn3sEZMsVkzs46SHpa0laQ7JJ0r\n6bqaJrUYOQXF4r6eq3WeAMqzdK0TQLOzQMm4+Y2kQYUPmtmPJPXNWa/QIZKWq2aCzcC/JK0raXpB\nfCtJq0o6092HFjz2R0kXSfq4+umV7HlJz9Xz2MTGSwNAJVEgoFSfS/pU0q/N7E/uvqDg8SMkmaSH\nJO1V+GR3n1z9FJs2d58j6Z2Mh1ZP//0k4zmfKvm+N0XPufvgWicBoLI4xYBy3Cipi6Rf5gbNbBlJ\nh0p6SdJbWU/M6kGwxKFm9pKZTTOzb81sipk9bmb7Z2yju5ldaWbvpev+18z+ZWZnN5S4ma1oZr83\ns2fMbKqZzUv3+aCZbVPPc7Y3s1Hp+t+Z2Wdm9rKZnVOw3mpm9v+Z2QQzm21mM9L/DzeztXLWy+tB\nMLNe6ffkb+kqtxSeollcD4KZbW1md6d5zUu/d9eb2eqF66brb25mj5nZLDObaWZPmdm2DX3vllT6\nM3MzuyTjsd+kjz1pZm1y4oeZ2T1m9qGZzU3zfdHMBtazj+fS7SxjZn8ysw/SMfKOmR2Zs95vzew/\n6Tanmtm5uftN1+mVbmu4ma1jZvenY222mY02s1+UePzdzeyv6bF8Z2ZfpuNuy4x1VzCzs83szfSY\nZ6XHcqeZbV7KfoFyMYOActwu6VIlswX358T3kLSapDMkrV3C9i5QMoX+kaT/J+lrSV0lbSlpP0l3\n1q1oZltIelzSypJekHSvklMW60kaLOn8Bva1brq/F5Sc7/9KUo80913NrL+7P5azv13S9WZKelDJ\nFP/K6XaOVdInIDNbTtKLkn4o6UlJo5TMpPSUtKekuyV9WE9OM9LtbJKu+4Ckcelj4+p5Tl1+v1ZS\nsH2X5jdF0o+U/Gz6m9k2ubM2ZvYTSU9Jaqvke/d+ut/nJD2zuH1VwGmSfiLpZDN7xt0fTnNaT9KV\nSmanBrr7opznXCvpbSU/r08l/UDSbpJGmNmP3b2+ovAOSVtLekTSfEn7SrrBzOZL2khJIfuQpKeV\n/Oz/JGmOpIsztrWmpH9KelPS9UrG5v6SHjWzA939zozn5DGzzSQ9oWTsPK7ke99ZySzbaDPb290f\nSdc1SY+l36t/Shqm5JTdGpL6SfqHpLEN7RNYYu7OF19FfUlySVPT/9e9aHXPefwxJX/cl5M0JF3/\nsIJtPJcMu7zYl5KmSlouY5+dc/7fVkkR4ZIOzFh3jYLliZImFsRWzN1mTry7kqn98QXxe9L9bdxA\nbv3T9S7LWK+tpBVylvul6w4uWO+wrO9Z+tjw9LFeObHekuYp+SPfrWD9n0laKOm+nJgpObXhkvYs\nWP/ENO6S+hU5Hgan6z+X/j/ra5uC56ytpNiaJqmbpGWV/OFdKOnnGfv4YT3fz6eV/OEvPO7n0pz+\nLWmlnPha6ffqq3QMdct5bCUl/SDTJC2dE++V8z35S8F+tkj3/5Wkjov7GSp5I/a+pG8l9S3YzupK\nis5PJbVLYxum27gv49jbSOpUjd9vvvgq/OIUA8p1o6SlJB0uSWbWU9JOkm715Bx7qeYr+SORx91z\nG/n6K3nRftDdb8tYd0pDO3H3rwu2WRefquRd/jpm1iPjqXMbyG1x681z91kN5VaGYyQtI+lEd89r\nXnT3Z5TMKPQ3sxXS8E8k/VjSC+7+QMG2/irpgzLz6CvpnHq+8k7buPv7ko5S8u75NklXS1pf0oXu\n/lThht095OTu89LnLS1px3pyOsPdZ+Q850NJo5UUA+fnfr/S9UalOXXL2NbXks4ryGGMpFvT7e1d\nTw51dlcys3SVuz9fsJ1PJP1ZySm7wmPJGkuL3P2rBvYHVASnGFAWd3/FzP4j6XAzG6JkSruNksKh\nVLdK+p2kt8zsLiVd8f90968L1qv7Y/NomWlLkszsp0reMW+r5FMDbQtW6Sapblr+VkkDJL1iZndK\nelbSi2lBket5Je8Ez0inkx9RcsphnLuHwqdC6voG+madx1ZybEspmWkYK2mznFzzuPtCMxut5A9Z\nqc71EpoU3f0OM9tRyZjpo+QP9zlZ66bF2h+U/PHsoWTGIVfWH3RJGpMRq2v+zJqerysYukuaVPDY\nq/UUeM8pOVWxqb7vH8lS93PqaRnXvlBySkhKTls9ouSUyjhJ/5cW3g8o+R6NSYsjoFFQIGBJ3Kjk\n3PEukn4taay7v1bGdk5W8u71cCX9C2dIWmBmj0g6NX3XKSXv1qQl+Kifme2tZKbgWyW9Ah9Imi1p\nkZKp/76S2tWt7+73mtkvJZ2a5nd0up2xkv7o7k+m681MmxzPVXJOe+d0E9PN7BpJQ9x9frl51+MH\n6b+/b2C9Dum/K6b/fl7Pep8tcUbFu1tJgSAl76xDEZU2dv5LUicl592fUPJufqGSmaRDlfOzypVR\nXErJKTGl26jvsWUyHmvo+7ViPY/Xqfs57dfAeh2k/xVrP1PSF7Gvvu+LmGVmf1My7r5pYFvAEqNA\nwJIYoeTF63ol7+TOW/zq2dI/DldIusLMVpW0naQDlLygrm9m67v7d0qa+aT63zUW43wl56K3cPfx\nuQ+Y2fVKCoTC/B6W9LCZLa+k8e2XSqb3HzKzTd397XS9qZJ+kzaZraekD+A4JS/0bSQ1+CmLEtX9\noVvR3WeWsP5q9TzeZclTapiZdZZ0k5KmQEm63MyedfdpBaueouSP66/dfXjBNv5PSYHQGBr6fmUV\nHLnqHt/T3R8sZofpaYSTlTR0rq1kXB4t6XglhfLBxWwHWBL0IKBs6bnbu5VMy85W8umGJd3mF+5+\nr7v/SklX/Q8lbZA+/HL6765LsIu1Jb2dURy0UVKYLC632e7+jLufImmoklMTIRdPvOXuVynpy5Ay\nrglRAXXfj+2LXP/V9N9QBJnZUmrg+CshLZ6GKynyTky/ukr6e/pYrrpPwtyTsalwDFW0WU4fR65+\n6b8NzZqV+nPK4+7vu/tNSo75GyWfdAGqjgIBS+osJU1aO5fTiGdm7cxsx8I/DpZcU2HldLHuneYo\nJZ9M2CN9B1m4rWJmFiZK+lHuNQLSfZ+j5F1/4TZ3NLPC897S9+8q56TrbWDZ90nIW6/C/qqkufMy\nM+td+KCZtTWz3D9KL0maIKmPmRX+kTle5fUflOoUJU17/8/dh7n7MCUfSdxF8VTJxPTffrlBM9tZ\n35+eaAwrKpkFys1hC0kHKZkduK+B5z+g5FTWcWa2W9YKZrZt+lFZmdmaZrZ+xmqdlJxSCc2LQDVw\nigFLxJPP2C/J1RGXVfK5/Ilm9oqSBrH2St55r6vkEwvj033NM7P9lJyLvs3Mjlby7qx9uu6OanhM\nX6bk/gavmdk9Sv7A/lRJcTBKySclcl0iqZcl9xSYqOT0xOZKTh9MUvLHTZJ+LulSM3tJyUcJv1Ay\ns7Knkv6Gv5TyTSmGu79jZodLullJg+djkt5Vch69h5J3rNMkrZOu72b2GyW9F/eYWd11EDZO839M\nyR/qUvWrp/lOkma4++WSlDZSXqjkY4ZH5axztJJrXlxgZi+4e9077muU9Lbclf6sPlYym7SLkutl\nhItoVckLko4ws62VNJ7WXQehjaSjGzq94+7zzWyAkusfPJyOkXFKisY1lBz7Wul25yj5edyX9rm8\nqaS5chUlY2kZZV+rAag4CgTU2mwlXeo7KPkY3l6SZil5x3WMkj9+/+PuY8xsEyWNjLumz5ml5A9d\nZid8wfOvN7PvJJ2k5Bz2XCUNcL+WtI9igTBUyQzJFkr+iC5SUhANlXR5zkfOHpd0uZKu/D0ldVTy\n2fYnJV3q7i8V9d0okbuPNLPXlTRR7iDpF0q+p58oOf1zZ8H6L6azChfo+9Mjryh5l76zyisQ+qr+\nKf9JSnoMVszJ5YDcJsK0wfMAJX987zCzTdx9hru/YWY7KLmmxm5KXq9eV/KpkhlqvALhI0m/VXIv\njN8qeRf/qqTz3P3xYjaQHsvGSmZQfqlkvC1SMkZeUzJ26z42O0ZJIdVXyc+jk5JCb6ykK919iT7F\nAxTL3BvlzrsA0Kykp4w+kvQ3dz+spskANUAPAgAACCgQAABAQIEAAAACehAAAEDADAIAAAgoEAAA\nQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAg\nAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQIAAAg\noEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAqEZMbPRZnZYrfMAKolxjZbGzEaa2eBa\n57Gkal4gmNk3OV+LzGxuzvJBNcyrnZm9Z2YTS3jOEDObn+Y+w8xeNLOtq5jm4nJpb2ZXmNknZvaV\nmV1lZkvXIpfWiHFdPWa2tpk9YmazzGy6mQ2tVS6tSVMb02Z2hpm9lY6DD83slBKee4SZLUxzn2lm\nr5nZbtXMdzG5dDOzUWb2qZm5mXWvRR5Zal4guHuHui9JkyX1z4ndWrh+I/6RO0PSp2U879b0WFaV\n9Iqke7JWaoTjOFPSxpLWl/RjSdtI+mOV94kU47o6zKydpCclPS5pNUlrSLq9mvtEoomO6YGSVpK0\nu6STzWzfEp77j/RYOkn6u6S7zGzFwpUa4TgWSXpEUim5N4qaFwgNSd+93Glmt5vZLEkDC6dvzOzn\nue+IzKy7md1nZtPM7CMzO67Efa4taX9Jfy43b3efJ+lvkrqZ2UppxfqCmV1pZv+VdFa6ryPM7J30\nXf6jZrZGTh67mNkEM/vazK6QZCWk0F/SFe7+lbt/IekqSYeXezyoLMZ12eP6N5ImuvsV7j7H3ee6\n+3/KPR5UTmOPaXe/yN1fc/eF7j5e0ihJPy01b3dfKOlmSctJWrMuRzMbZGafSboxzXUPM3s9nUUb\nbWYb5BzH5mY2Lp3NuF1SuxL2/6m7XytpbKm5V1uTLxBSe0u6TdKKku5c3IpmtpSkhyT9W1I3STtJ\n+r2Z7Zg+3tfMpjewv79K+oOkb8tNOH2nc5iSF7MZafgnksZLWkXSxWm1+3tJe6axV5Qcp8xsVUl3\nK3nH11nSVElb52x/zXSgrl5fCsp/4TVJvcysQ7nHhIpjXJc+rreRNNnMHrfk9MIzZrZ+uceDimvs\nMV23rTaStpP0VqkJpzMEv5E0S9IHabi7pA6Sekg61sy2VFIoHCHpB0oKigfMrG36O/FAGls5/f9e\nuceZjultSs2t1ppLgTDa3Ue5+yJ3n9vAuttI6ujuQ919nru/L+kmSQdIkrs/7+6d63uyme0naYG7\njyoz1wPNbIakKZI2VPILU2eyu1+bVrxzJR0taai7T3D3BZKGSNrKzLpJ+qWkce5+n7vPl3SJpGl1\nG3L3j9x9JXf/pJ48HpV0kpl1NrOukn6Xxpct87hQeYzr0sd1d0n/lz5vdSWnGx4ws2XKPC5UVqON\n6QLnS1qg5FRBsbZLx/RnSqb393L3WeljCyQNTvOaK+koSde4+7/TcX5zut6WSmYtXNJV7j7f3e+Q\n9FrdTtL1V3L3l0vIrUloLk1rU0pYt6ekHukPvs5Skp5r6Inpu+sLJe1cUnb5bnP3w+p5rPA4ekq6\nOp1mrbNIyYvg6rnru/siM5taQh7nKXkRfV3JO8ablfQjFFWRo1Ewrksf13MlPe/uT0iSmV2s5LRG\nb5Xx7hEV1yhjOpeZnaikqNg+PQVWrNHu3q+exz4v2FZPSQeZ2ck5sbZKZj7aSprq7p7z2KQS8miy\nmkuB4AXLs5WcL6rTJef/UyS95+7rlrGfdZRMKb1oZlLyg18xPQ+1pbuXMvizFB7HFElnu3uYikvP\nb+2Ss9xGyQtscTtynyPpmPRLZnaspDEFgxi1xbgucVxLekPS5ovZN2qrsca0JMnMjpJ0qqQ+i5l1\nKkfWmD7X3S/OyGFHxTHcQy2gYG0upxgKjZO0u5l1SqfPT8h57J+S5pnZqZZ81G8pM9vQzDbP3lTY\nbg9Jm6RfR0v6JP3/J5JkZlPNbGCFjuM6SWea2brptley77twH5K0iZntmZ4jO1nJ+dyipM0/Xc2s\njZn9RMmnGgZXKG9UB+O6YSOUTA3/LD2HfZqkjyVNqFDuqKxqjWmZ2aGSzpW0k7tPzHh8tJmdVYFj\nkKQbJB1nZltaooOZ9Tez5SWNltTGzI43s6XT03mblbJxM2uv7xsb26V9DTXXXAuE4UqaoiZJekzS\nHXUPpOc8d5O0laSJSqbUr5fUUZLMrF/BlJZyn+vun9V9SfpK0sJ0eWH6Q+ykpOlqibn7XZIuVfLx\nmplK3h3tnD72uZKO879I+lLJC/z/9mtma1nyGd76mrl+JOllSd8oOb1wmrs/XYm8UTXDxbhe7Lh2\n97clHSppWHocuyk5d7ygErmj4oarCmM6NURJw+BY+/56DH/Neby7pBcrcRDu/oqS2dhrlYy7d5V8\nxFLu/p2Snpwj08cGSLq/7rlp4fONmW2bte20UJ4rqe5Y31cy81Jzxoxz8cysn6TfuPvBtc4FqBTG\nNVoaM+slaYS7b1/jVJo1CgQAABA011MMAACgiigQAABAQIEAAACCkq6D0LlzZ+/Vq1eVUkFzN3bs\n2OnuXspH1mqOMY3FYUyjpSllTJdUIPTq1UtjxowpLyu0eGbW7K4expjG4jCm0dKUMqY5xQAAAAIK\nBAAAEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAUNKFkgC0PnPmzAmxAw44IMTWWmut\nvOXLL7+8ajkBWd58880QO+6440Jszz33DLFTTjmlKjk1Z8wgAACAgAIBAAAEFAgAACCgQAAAAAFN\nigAWa+rUqSE2atSoEFt22WXzls8555ywTqdOnSqXGFDgmGOOCbHRo0eH2AsvvBBiWbfIHjBgQEXy\naq6YQQAAAAEFAgAACCgQAABAQA9ChV133XUhlnVe7J577slbbu3nutD8rbbaannLbdu2rVEmaK32\n33//EHvxxRdDzN1DLOvCXq39dZkZBAAAEFAgAACAgAIBAAAEFAgAACCgSXEJXHPNNSF2/PHHF/Xc\nFVZYodLpADW166675i0vv/zyNcoErVXW62/WXRrnz5/fGOk0e8wgAACAgAIBAAAEFAgAACCgQAAA\nAAFNikV66aWXQuzEE08MsXbt2oXYyJEjQ2ynnXaqTGJAlV177bUhljXOTzrppMZIB6jXM888E2IL\nFiyoQSYtAzMIAAAgoEAAAAABBQIAAAgoEAAAQECTYoa33347xA444ICinvvnP/85xPbZZ58lzglo\nDJMnTw6x4cOHh9hyyy0XYr17965GSkDRssZv1q2dURxmEAAAQECBAAAAAgoEAAAQUCAAAICg1Tcp\nTpw4McR23nnnEPvkk09C7LLLLgux3/3udxXJC6iFp556KsRmzJgRYhdddFFjpAOUJOv1PMvSS8c/\nfZdcckmFs2n+mEEAAAABBQIAAAgoEAAAQECBAAAAglbXpFh4689TTjklrPPxxx+H2MknnxxiJ5xw\nQtl5LFy4MG+5TZtYq5lZ2dsHGvLFF1+EWNaVQLt06RJihx12WDVSApbI008/XdR6nTp1CrEtt9yy\n0uk0e8wgAACAgAIBAAAEFAgAACCgQAAAAEGra1K8/PLL85bvu+++sE7WrZ2X5CpbixYtanAfWVdv\nPOKII8reJ9CQRx99NMQmTJgQYvvtt1+IrbbaaiE2d+7cvOXChmBJWmGFFUpJEaiKP/7xj7VOoVlg\nBgEAAAQUCAAAIKBAAAAAQYvuQZg0aVKIXXnllXnLG264YVjnnHPOqWgeU6dODbG77747b3n8+PFh\nnYEDB4ZY+/btK5cYWo3Zs2eH2N///veinnv66aeHWFZ/QWFfzeeffx7WeeSRR0Js5ZVXLioPINdN\nN90UYqNHjw6xZZddNsR23XXXquTU0jCDAAAAAgoEAAAQUCAAAICAAgEAAAQtuknxoosuCrEpU6bk\nLZ966qlhnXXWWafsfc6fPz/EzjzzzAafl3XhGRoSUSmXXXZZiD3zzDMhtsMOO4TYFltsEWJPPPFE\niD344IMN5lH4+yfRpIjy3HPPPUWtt88++4TYkrzGtybMIAAAgIACAQAABBQIAAAgoEAAAABBi2lS\nfP/990PslltuCbHdd989b/mEE06oaB5ZTVgjR45s8Hl77bVXRfNA6/Xmm2+G2A033FDUcw8//PAQ\nmz59eoj97ne/a3BbXbt2DbEuXboUlQeQ61//+leIPf3000U9d8CAAZVOp9VgBgEAAAQUCAAAIKBA\nAAAAAQUCAAAIWkyT4qhRo0Lsu+++C7FFixZVNY+77rqrrOftt99+Fc4ELVHWlTofe+yxvOVjjjkm\nrPPxxx8Xtf2shq7HH388xN59990Gt7X00vHlJes20Vm/p+3atWtw+2i5Zs6cmbec1RQ7b968ENt7\n771DjAbw8jGDAAAAAgoEAAAQUCAAAICAAgEAAAQtpklx5513DrGsJqlHH300bznrVqCDBg0Ksaxb\n3r733nshNmTIkMXmWeeII47IW1511VWLeh5aj6+//jrEspqwnn322Yrtc/nll6/YtrKuKtq9e/cQ\n69GjR4gNGzYsxHbaaafKJIYm74orrshbzrqSYpas3w8zq0hOrREzCAAAIKBAAAAAAQUCAAAIKBAA\nAEDQYpoU11tvvRA755xzQuzss8/OW77vvvvCOllNX1tttVWIvfTSSyH2zTffhFibNrEOO//88xtc\nB61HVkPiaaedFmLFNCR26NChqG117NgxxG6//fYQ+/e//93gPpdEVjPxq6++GmI0KbZMWVfBHTx4\ncIPPy2pu7dOnTyVSQoq/SgAAIKBAAAAAAQUCAAAIKBAAAEDQYpoUs5x11lkhtvbaa+ctn3766WGd\nrCvAPfHEE2XnkdU406VLl7K3h+Yt65bNWU2EWVcTLMa5554bYqecckqIffvttyFW2Dxbn6yr0228\n8cZ5yz/72c/COv379w+xzTbbLMSyGijR/L3xxhshdtRRR4XYokWL8pZ33XXXsM7Pf/7zEMtqbs2K\nlSurWb1bt24V235TwwwCAAAIKBAAAEBAgQAAAIIW3YOQ5YADDshbHjBgQFhn4cKFIfbWW2+F2JZb\nbhliWXfDGz58eAkZoqXLugtouf0GknTwwQfnLZ944olFPe/OO+8Msa+++qqo5+6yyy4h9sgjjxT1\nXLQ8hT0DkvTiiy+G2K9+9asQ++yzzxrcfuFdeOuLVdsKK6wQYocddliIZd1VMqsXbamllqpIXtXC\nDAIAAAgoEAAAQECBAAAAAgoEAAAQtLomxUJt27Ytar2suzRm6dq1a4j17NmzpJzQsv35z38u+7lr\nrrlmiBVe3KjYxqdp06YVtd4hhxwSYrfccktRz0XLM3369BA78sgjQ+z+++9vjHTyFF4IT5K+/PLL\nEPvRj34UYll31P3uu+/yll977bWwzlVXXVVULOtiT4MGDQqx7bbbLsRqhRkEAAAQUCAAAICAAgEA\nAAQUCAAAIGj1TYrFKraxbOedd65yJmhOshqknn322aKe265duxC7/fbbQ6zcJtiPP/44xNq3bx9i\n+++/f4hlNXShdbj00ktDrNiGxKWXjn9y1ltvvRA78MAD85Y33XTTsM7WW28dYssuu2yIZV3lMSuP\nLIVNillXh8y6W+SFF14YYllXfnzyySdD7Iwzzgixo48+Om+5e/fuMdkq4LccAAAEFAgAACCgQAAA\nAAEFAgAACGhSzDBu3LgQe+qpp2qQCZq7+fPnh9i3335b1HMffvjhEMtqzCrX6aefHmJZV03MahBD\n63XEEUeE2HXXXRdim2yySYhlNbwWNuA1JYXNjL/4xS/COlmxrNs9ZzUkDh06NMSuvvrqECu88mPW\n72k1MIMAAAACCgQAABBQIAAAgIACAQAABDQpZsi6tXNWs1mHDh1C7KCDDqpKTmieunTpEmKff/55\nDTKJsm5NnhUDcq211loh9t///rcGmTRdP/7xj4uKHX/88Y2RTtmYQQAAAAEFAgAACCgQAABAQIEA\nAAACmhQzrLrqqiGWdRvRzTffPMS23XbbquQEAEBjYgYBAAAEFAgAACCgQAAAAAEFAgAACGhSzNC7\nd+8QmzNnTg0yAQCgNphBAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgAACCg\nQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQIAAAgoEAAAAABBQIAAAjM3Ytf\n2WyapEnVSwfNXE93X6XWSZSCMY0GMKbR0hQ9pksqEAAAQOvAKQYAABBQIAAAgIACAQAABBQIAAAg\noEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAA\nAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCoRmxMxG\nm9lhtc4DqCQzG2lmg2udB1ApLWVM17xAMLNvcr4WmdncnOWDapDPEDObX5BXjzKeO8PMXjSzraud\ncz25HGRmE8zsazP73MxuMbMOtcilNWpq4zrNaQsz+0eaw2dmdnyRzzvCzBamz5tpZq+Z2W7Vzncx\n+Qw0s0lpPvea2Uq1yqU1aWoEfMy5AAAgAElEQVRj2sxONbOP0jH5sZldYmZLF/ncJjOmzezn6ffz\nm1p+P7PUvEBw9w51X5ImS+qfE7u1cP1iB8ASujU3L3efXOpzJa0q6RVJ92St1AjH8Q9JP3X3FSWt\nLWlZSedVeZ9INbVxbWarSnpE0jWSVpbUW9JTJWziH+mxdJL0d0l3mdmKGfup9nFspOQYDpLURdJ8\nSX+t5j6RaGpjWtL9kjZx946SNpa0haRjS3h+kxjTqckFf3PC97MWal4gNCR9V36nmd1uZrMkDSyc\nvkkrsIk5y93N7D4zm5ZWmMc1dt7uPk/S3yR1M7OV0or1BTO70sz+K+msNNcjzOwdM/vKzB41szVy\njmOXnFmAKyRZCfuf7O7Tc0KLlBQKaAJqMK5Pk/Swu9/u7vPcfaa7v1Nq3u6+UNLNkpaTtGZdjmY2\nyMw+k3RjmuseZvZ6OpM22sw2yDmOzc1snJnNMrPbJbUrIYWBku5399Hu/o2kP0naz8yWK/VYUFmN\nPabd/QN3/zonVNZrXBMY001Wky8QUntLuk3SipLuXNyKZraUpIck/VtSN0k7Sfq9me2YPt7XzKYv\nZhOStJeZ/dfM3jSzo8tJ2MzaSTpM0kR3n5GGfyJpvKRVJF1sZvtK+r2kPdPYK0qOs+4d392SzpDU\nWdJUSVvnbH/NdKCuvpgc+prZ15JmStpD0uXlHAuqpjHH9TaSZpjZy2b2hZk9YGbdS004fTf1G0mz\nJH2QhrtL6iCph6RjzWxLJS+qR0j6gZIX3wfMrG36e/FAGls5/f9euceZjutt6klhfUmv1y24+wQl\nfxh+VOqxoCoa9bXazA5Oi5FpSsbGDaUm3ATGtCR1teRU8IeWnCppEgVvcykQRrv7KHdf5O5zG1h3\nG0kd3X1o+k7pfUk3STpAktz9eXfvvJjn3y5pXSV/sH8r6Twz26+EXA80sxmSpkjaUMkvTJ3J7n6t\nuy9Mj+NoSUPdfYK7L5A0RNJWZtZN0i8ljXP3+9x9vqRLlPwSKD2Oj9x9JXf/pL5E0mNdUdIakv4/\nJdOCaDoac1x3l3SopOOUvOh9LKmUaczt0nH9maR9Je3l7rPSxxZIGpzmNVfSUZKucfd/p2P95nS9\nLSX9VJJLusrd57v7HZJeq9tJuv5K7v5yPXl0kPR1QWympBVKOBZUT2OOabn7CHdfQdI6kq6X9EUJ\nuTaVMf2WpE0kdVVSJG0j6S8lHEfVNMa5lUqYUsK6PSX1SH/wdZaS9FwxT3b3t3IWR5vZVUoGz11F\n7v82dz+snscKj6OnpKvT0wd1Fil5MV89d313X2RmU4vMIY+7TzWzp5RU9luVsw1URaONa0lzJT3t\n7mMlyczOlfSZmXVIp+obMtrd+9Xz2OfpKbXcXA8ys5NzYm2VvEtsK2mqu3vOY5OKPAZJ+kZSx4JY\nRyXv/lB7jTmm/8fdJ5jZBCX9KL8q8mlNYky7+6eSPk0XPzCzPyjpXWv0U+OFmkuB4AXLs5WcL6rT\nJef/UyS95+7rVnDfRZ/7L2JbuaZIOtvdw1Rcen5rl5zlNkoKh3ItLemHS/B8VF5jjus3MvZXuFyu\nrHF9rrtfXLhiOn1cOI57KHkXVYy3lDSk1W2vt5KZ0PeKzhbVVMvX6kq+xjXmmM7ad6X+5iyR5nKK\nodA4SbubWScz6yrphJzH/ilpniUfgWmfnv/Z0Mw2L2bDZraXJU2FZslHFI9Xck6p7vGpZjawQsdx\nnaQzzWzddNsrpX0JUnJubhMz2zM9R3ayktMeRbHko2BrpP/vJel8SU9XKG9UR9XGtaRbJO1rZhuZ\n2TKSzpT0fN3sQdp0dVaFjuMGSceZ2Zbp71EHM+tvZstLGi2pjZkdb2ZLp6fvNith2yOV9Aj9JN3e\neZLucvc5FcodlVXN1+oj014tmdn6kv6gnNe45jKmzWyHnNfqHpIuVM7fnFpqrgXCcCXNfpMkPSbp\njroH0nP5uymZSp8oabqSc1MdJcnM+hVMaRU6UNKHSqYs/yZpSN1HTsysvZKPxLxSiYNw97skXark\n4zUzlbzL2zl97HNJ+ys5F/Wlkor0f/s1s7Us+bxsfU2KG0p62cxmKxnAbynpeUDTNVxVGtfu/oSS\njv9HlZyn7ankEwF1ukt6sRIH4e6vSDpG0rWSvpL0bt2+3P07JX05R6aPDVDycTWlx7FUOq63rWfb\nbygp2u9Ij6OdpN9VIm9UxXBV77W6j6Q309e4hyQ9KOnsnMebxZhW8vHMl81sjpLX6leVvCGsOcs/\nbYLFMbN+kn7j7gfXOhegUtIZphHuvn2NUwEqgjFdGRQIAAAgaK6nGAAAQBVRIAAAgIACAQAABCVd\nB6Fz587eq1evKqWC5m7s2LHT3b3oj2I2BYxpLA5jGi1NKWO6pAKhV69eGjNmTHlZocUzs1KuiNck\nMKaxOIxptDSljGlOMQAAgIACAQAABBQIAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAA\nAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAISrqbIxr27rvvhtjRRx8dYgceeGDe8pFHHlm1nAAAKBUz\nCAAAIKBAAAAAAQUCAAAI6EFYAln9BrvvvnuIffjhhyE2ceLEvGV6EAAATQkzCAAAIKBAAAAAAQUC\nAAAIKBAAAEBAk2KRrrjiihC7/PLLQ2zy5MlFba9nz55LnBOQ5corrwyxE044oQaZoLkbOnRoiJlZ\niG2wwQYh1r9//6rkhMbDDAIAAAgoEAAAQECBAAAAAgoEAAAQ0KSYYcGCBSH29ttvh9ikSZNCLKuB\np3fv3iE2cuTIMrNDazZ79uwQO+OMM/KWP/roo7AOTYoox5lnnhliWa9xyyyzTIi1b9++KjmVyt1D\nbPDgwSHWtm3bBrf1xBNPhNjBBx8cYvvtt19xyTVxzCAAAICAAgEAAAQUCAAAIKBAAAAAAU2KGa67\n7roQGzZsWNnb69y5c4h179697O2h9cpqQLz66qvzlv/1r381VjqAJGn+/PlFxWohq0nxtNNOq9j2\np02bFmI0KQIAgBaLAgEAAAQUCAAAIKBAAAAAQatvUvzkk09C7KabbgqxrEaXrFiWv/zlL6UnBmQ4\n8cQTQ2zDDTfMW24qV7BD83fttdeGWNbVBIs1ffr0EBs9enTZ20N1MYMAAAACCgQAABBQIAAAgIAC\nAQAABK2+STHrls1vvPFGiGXd4jTLHnvsEWKbbbZZ6Ymh1XvyySdDbOHChSH2+uuvN0Y6eT744IMQ\nmzFjRt7y5ptvHtZ59tlnQ+zFF18sO4+NN944xPr371/29pDvt7/9bVGxYj311FMh9otf/KLs7RWj\nV69eIfajH/2oqOeuv/76ecs/+MEPwjoDBgwoK6/mgBkEAAAQUCAAAICAAgEAAAQUCAAAIGj1TYor\nrLBCiGXdnjnrCmBZ/vnPf4bYu+++G2IbbLBBUdtD6/XYY4+FWJs2lavps64iutdeexX13JkzZ4bY\nd999l7fcrVu3sE7W71HW70exVllllRDr2bNn3jK3v246pk6dWrFtLb10/PM1aNCgEDv44IND7Ic/\n/GHF8mjJmEEAAAABBQIAAAgoEAAAQNDqexCyegGyzsMOGzasqO1lnWO95pprioqh9crqB8i6YFfW\nOBwzZkzeco8ePcI6q666aogdfvjhIZbVW5B119L33nsvxAodeuihIZZ1oacLLrigwW3VZ9q0aSG2\n1VZblb09VM6sWbNC7LLLLitrW1kXKLrllltC7Je//GVZ20c2ZhAAAEBAgQAAAAIKBAAAEFAgAACA\noNU3KWY566yzQqzYJsUso0aNCrGjjz46bznrrnRoPQYOHBhizz33XIgVjhtJmjx5ct7yrbfeGtbJ\nalJcfvnlQ+yuu+4Ksawmxc8//zzECvXp0yfEpkyZEmJ33313iH300UchNm/evBDbZZddQuzmm29u\nMDdU3+jRo0NswoQJZW3r22+/DbGssZoV69u3b4gddthhIVbJi5C1FHxHAABAQIEAAAACCgQAABBQ\nIAAAgIAmxQxrrLFGiJ144okhVuxVwT7++OMQ22OPPfKWJ02aVGR2aO5eeeWVECu8GqIkbbrppiF2\n0UUXhdgll1ySt7zyyisXlcc999xT1HqVtPbaa4dY1t0WjzvuuBAbOXJkiHXt2jXEsu7wiMa36667\nhlhWA/if/vSnBrc1e/bsEBsxYkRRefz9738Psawxt8wyy4TYSSedlLecNbaWW265EMu602RzxAwC\nAAAIKBAAAEBAgQAAAAIKBAAAELSMTopGcM4554TYFltsEWJZV7qbM2dOiH322Wd5yyeccEJYJ+t2\nvJtsssli80TTd/3114fYN998E2JZV1fcfPPNQ+y2226rTGI1knVVxqyGRDR/WWM6q6Gv8Gqgb731\nVkXzuOGGG4pa7+qrr25wnQMOOCDEspox11tvvaL22ZQwgwAAAAIKBAAAEFAgAACAgAIBAAAElnUr\n1/psscUWnnXFN3xv7733DrGs2/bOnDmzwW2tttpqIfb666+HWFO5cpyZjXX32LnZhFV7TJ933nkh\nNmTIkBDbdtttQ+zpp58OseZ+hbbBgweH2MUXXxxixxxzTIgNHTo0xJZaaqkQy7oiXrkY003XAw88\nEGJZVyl96aWXQuz555+vSk6L86tf/SrEttpqqxA79dRTq5pHKWOaGQQAABBQIAAAgIACAQAABBQI\nAAAgaN4dT03QfffdF2JZV8479thjG9xW4dUWJWnevHnlJYaayLoCp5mFWJs2sVZv7g2JgwYNCrEn\nn3wyxP7whz+E2M477xxi7du3r0xiaBH23HPPomJZr5nfffddiGU1y44bNy5v+ZFHHiklxTx33XVX\niI0aNSrE2rVrF2LHH3982ftdEswgAACAgAIBAAAEFAgAACCgQAAAAEHz7oJqJjbaaKNap4AmLuvK\nmllNql26dGmMdBqUdaW+a6+9Nm95xIgRYZ2uXbuG2CGHHBJia6211hJkB3yvbdu2RcWyrnA6d+7c\nvOWvvvoqrDN16tQQy7pq4uTJk0Ps22+/DbETTjghxGhSBAAATQYFAgAACCgQAABA0GJ6EIq9O1ff\nvn2rmseNN94YYll3oSvmLpql3GkTzdtrr70WYoceemiI3X777SG28sorVyyPN954I8SyLvDy5z//\nOcR22223vOWsOzf26dMnxOg3QFO17LLLLnZZkj788MMQy+q1yepBaOqYQQAAAAEFAgAACCgQAABA\nQIEAAACCZtmk+Mknn4RY1l28shqivvjii7L2+eCDD4ZYVmPk559/HmILFiwIscI7+m2yySZF7bOp\nXCgHxVl77bVDbNq0aSGWdaGkrDsfHnDAASF2zTXX5C3//ve/D+u8//77i81zcXlkXbhl7NixIbb6\n6qvnLVeyeRLNz7Bhw0Isq4l7/fXXD7Gbb765Kjktzj/+8Y8QK7zrY+HvmiQ988wzIZb1e1SsrNeM\nWmEGAQAABBQIAAAgoEAAAAABBQIAAAiaZZPiwoULQ2zWrFkh9tBDD4XYww8/XLE8sq50WNh8KEkd\nO3YMsYsvvjhvuX///mGdrKtxoXl57733Quzwww8PsQ4dOoTYyy+/HGJPPfVUiPXu3bus3JZZZpkQ\nO/HEE0Nsxx13DLENNtigrH2iZZo+fXqIXXDBBSE2adKkEHv33XdDrLDBfL311isqj8cffzzEnnvu\nuRBr0ya+Nx49enSIFTYpVtryyy8fYlnNybXCDAIAAAgoEAAAQECBAAAAAgoEAAAQNMsmxaWWWirE\nshoBv/7666rmscYaa4TYpptuGmJZjV877LBDVXJC03fGGWeEWNYtj7NuD7vHHnuEWNbVO4tx7rnn\nhtixxx5b1rbQumU1+H322WdFPTfrdTqrkbdcxTaTl6t9+/Yh1qtXrxBbccUVQ+zMM88MsZ49e1Yk\nr0pgBgEAAAQUCAAAIKBAAAAAAQUCAAAImmWTYuFtZSXp/vvvD7HXXnutwW1deeWVIdavX78Q22ij\njULspJNOanD7QKFir3yY1bj45ptvVjodYInttddeIbb11luH2Pjx40Ms6/bntZB1VcNOnTrlLR9z\nzDFhnY033jjEdtttt8olVkPMIAAAgIACAQAABBQIAAAgoEAAAABBs2xSzNK3b9+iYoVoNASAysu6\nzfKUKVNCbM899wyxDz/8sMHtZ11VdKuttiouuQzrrrtuiGXd6rw1YQYBAAAEFAgAACCgQAAAAAEF\nAgAACFpMkyIAoGlbY401QuzVV1+tQSYoBjMIAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQ\nUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAAAAIKBAAAEFAgAACAgAIBAAAEFAgA\nACCgQAAAAIG5e/Erm02TNKl66aCZ6+nuq9Q6iVIwptEAxjRamqLHdEkFAgAAaB04xQAAAAIKBAAA\nEFAgAACAgAIBAAAEFAgAACCgQAAAAAEFAgAACCgQAABAQIEAAAACCgQAABBQIAAAgIACAQAABBQI\nAAAgoEAAAAABBQIAAAgoEAAAQECBAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAI\nKBCaETMbYmbDa50HUCmMabREZjbSzAbXOo8lVfMCwcy+yflaZGZzc5YPqkE+7c3sRjP7wsz+a2YP\nmtnqRT735+kxfGNms8zsHTM7tNo5Lyaf5c3sOjP70sy+NrNna5VLa8KYrg4z+4mZPZUewzQzu9PM\nVqtFLq1RExzXZ5jZW+m4/NDMTinhuUeY2cI095lm9pqZ7VbNfBeTy9kF39u5aW6dapFPrpoXCO7e\noe5L0mRJ/XNitxaub2ZLVzmlUyRtLmkDSd0kfSPp8hKePzk9lo6SzpR0k5n9uHClRjgOSbpJUgdJ\nP5a0sqTTGmGfrR5jumo6SbpWUk9JvSR9q2SMoxE0wXEtSQMlrSRpd0knm9m+JTz3H+mxdJL0d0l3\nmdmKhStV+zjc/fyC7+0lkp5296+qud9i1LxAaEg6BXmnmd1uZrMkDSycvknf5UzMWe5uZvel7zI+\nMrPjStjlmpIec/cv3H2upDslrV9q3p64R9IsSeua2dpm5mb2azObLOmJNNefmtnLZjbDzMaZWZ+c\n41jLzP6RVsiPS/pBsfs3s/Ul7Srpt+4+3d0XuvvYUo8DlceYLm9Mu/vD7n6Pu89y99mSrpb001KP\nA9XR2OPa3S9y99fS17bxkkapjPHg7gsl3SxpOUlr1uVoZoPM7DNJN6a57mFmr6fjerSZbZBzHJun\nY32Wmd0uqV2peaTbMUkHS/pbOc+vtCZfIKT2lnSbpBWVvLjVy8yWkvSQpH8rebe0k6Tfm9mO6eN9\nzWz6YjYxTNL2ZtbVzJaXdKCkR0tN2MzapNVsB0n/yXmoj6R1JO1uZmtIelDSOUre4Z8h6V4zq3vR\nvEPSy5I6S7pIycDJ3cdbZvarelLYWtKHki4ws+lm9oaZ7VXqcaBqGNOlj+lCfSS9VepxoKoac1zn\nbquNpO1UxnhIZwh+o6Tw/SANd1cyzntIOtbMtlRSKByhpKi9WdIDZtbWzNpJeiCNrZz+f6+c7S+V\nFhXbFJHODkpmNO4r9TiqoTGmgCphtLuPSv8/Nymy6rWNpI7uPjRdft/MbpJ0gJJpm+eVvDjVZ4Kk\nTyV9ImmhpNclHV1Crj3MbIakRZImSTrI3T8ws7XTx89x9zmSZGaHSHrQ3R9PH3vMzF6XtIuZ/VPS\nJpL6uft3kp41s0dyd+Tui3sX2D19/r2SVlfyyzPKzDZ193dLOB5UB2O69DH9P2a2qZLTHbuXcByo\nvsYc17nOl7RAyamCYm2XjusFkt6VtJe7z0pzXiBpsLvPkyQzO0rSNe7+7/S5N5vZmZK2VDJb4JKu\ncneXdIfl9EOkMxQrFZnToZL+X93vU601lwJhSgnr9tT3L2h1lpL0XJHPv17JzMrKkuZIGiTpYRU/\ndTXZ3Xst5vHcY+kp6f/MbO+c2DKSHlPyR/3LgoEySdIqReYxV8k52qHpAH3GzF5QUqVTINQeYzpR\nypiWJJlZbyX5H+fuL5XyXFRdY45rSZKZnaikqNi+7g96kUa7e796Hvu8YFs9JR1kZifnxNoqmflo\nK2lqWhzUmVRCHpKSpnJJ+yg5NdwkNJcCwQuWZys5X1SnS87/p0h6z93XLXNfG0s6ta5BxMyukvQn\nM1vJ3Wcs/qkNKxhEUyTd4u7HFK5nZj+U9AMzWzY9bywl011zC9etxxt1uyw7WVQTYzpRypiWma0p\n6Sklsxa3LUHaqI7GHNd17+xPldTH3T8pdzsZCo9jiqRz3f3ijBx2VDJjm6uHSj/dsa+kzyWNLvF5\nVdNcehAKjVNyvrOTmXWVdELOY/+UNM/MTrXk411LmdmGZrZ5kdv+t6RDzayjmS0j6Vgl76BmSP/7\nfOuwCh3HCEl7m9lOaZ7tzWwHM1vd3T9Q8kd+cHqeq49Km059VtJnkv5gZkunz99OaSMZmhzGdAPS\n/oZnJF3q7jdWKF9UV9XGtSUftz1X0k7uPjHj8dFmdlYFjkGSbpB0nJltaYkOZtY/fdc/WlIbMzs+\nfa3dT9JmZezjUEl/Kyi4a6q5FgjDJY1XMo3zmJLGJ0mSuy+QtJukrSRNlDRdyRRrR0kys34FU1qF\nTlFynvZ9SdOUTMkPyHl8DUkvVuIg0kG9t6Sz031NVlIN1/1cDlAyDfxfJedbR+Q+38wmmNn+9Wx7\nnqQ9JO0p6WslHw87yN3fq0TuqLjhYkwvdkxLOkrJxxuH2PefGV/iGRBU1XBVb1wPUdIwODZnPPw1\n5/Huqty4fkXSMUpeR79Scpp2YPrYd0rG/JHpYwMk3V/33LTw+cbMtq1v+2bWQ0nT7Yj61qkFa0LF\nSpNnZu0lvSZpw3RwA80aYxotkZn1kjTC3bevcSrNGgUCAAAImuspBgAAUEUUCAAAIKBAAAAAQUnX\nQejcubP36tWrSqmguRs7dux0dy/poje1xpjG4jCm0dKUMqZLKhB69eqlMWPGlJcVWjwzK/nqYbXG\nmMbiMKbR0pQypjnFAAAAAgoEAAAQUCAAAICAAgEAAAQUCAAAIKBAAAAAAQUCAAAIKBAAAEBAgQAA\nAAIKBAAAEFAgAACAgAIBAAAEFAgAACAo6W6OaNjYsWND7Oc//3mIrbTSSnnLjz/+eFind+/elUsM\nAIASMIMAAAACCgQAABBQIAAAgIAehCLNmTMnxI4++ugQe+ihh0Js5syZDcb222+/sM7rr79eSorA\nEjOzEBswYECIuXuIrb/++nnL559/fuUSA9DomEEAAAABBQIAAAgoEAAAQECBAAAAApoUM/znP/8J\nscMPPzzEXn311RDLat7Kavwq1K9fv+KSA6ooa6zef//9IZY1zh944IG85U033TSsk9XwCBSaPXt2\n3vI777wT1rnxxhuL2tYXX3wRYllj+qijjmpwWwceeGCI9enTp6g8miNmEAAAQECBAAAAAgoEAAAQ\nUCAAAICg1TcpfvLJJyF2+eWXh1hWQ2Il3XLLLSG25ZZbhtjAgQOrmgdat+uuu66o9c4666wQmz59\net7yhRdeGNahSbF1GzJkSIgVNrdKsUlxwoQJYZ1iG8KLXe+GG25ocL2su/U++uijIda5c+cQa46Y\nQQAAAAEFAgAACCgQAABAQIEAAACCVt+keNFFF4XY8OHDGz2PwqYcSTr00ENDLKtZcpNNNgmxQw45\npDKJoVUp5mpyUvY4LPbKdmh5Ro4cGWKnnnpqiGVd1bCYxsJ11103rNOzZ88Q23vvvRebZ51ib2G+\n1VZb5S2PGTMmrDN58uQQo0kRAAC0WBQIAAAgoEAAAAABBQIAAAhaXZNi4ZWwRowYEdbJalYp1pI8\ntxiXXXZZiO2///4hRpMiGlvh2N9+++1rlAkaW+FVNOuL/fa3vy1qe0ceeWTe8jrrrBPWWW655YrM\nrjjjx48PscJjyGqobMmYQQAAAAEFAgAACCgQAABAQIEAAACCVtek+Ne//jVveebMmWGdYhtRsq5g\n+OCDD4bY66+/HmKFt5R++umni9pnlocffjjEsm4f/etf/7rsfQC57rvvvhAr/L0p9qp2aP5OOumk\nomJN2Zw5c0Ks8Aq3WVdIbClXTczCDAIAAAgoEAAAQECBAAAAAgoEAAAQtLomxb/97W95y8U2JBbe\n9lOS7r333hDr2rVriHXr1i3EOnXqlLe8JE2KWVcUW2211creHtCQrAbEG264IW+ZKymiOSmm8ZYr\nKQIAgFaPAgEAAAQUCAAAIGh1PQjlOuKII0Js2WWXDbFZs2aF2IwZM0Js2LBhlUlM0s9+9rMQ2223\n3Sq2fbQe06ZNC7ELL7wwxLLO16633npVyQloDG+//XaIFd6htEePHmGdrFixJk2aFGJZd8Es3Mcq\nq6xS9j5LwQwCAAAIKBAAAEBAgQAAAAIKBAAAENCkWKTTTjstxK677roQ69ChQ4i98MILVcmpzh57\n7FHV7aP5y2qGymp0GjlyZIgV3nlUyr441/PPP19mdkDt3X///SFWeGGkrIt/FV4grD5Zjb2vvvpq\niBXTpHjppZeGdapx91RmEAAAQECBAAAAAgoEAAAQUCAAAICgRTcpnnLKKSFWeGWsYn399dchNnbs\n2KK2X8k7gGU1uuy5554V2z5apqy7kV5yySUhdtFFF4VY1vgdNGhQiK2zzjplZgc0riFDhoRYMX8b\nLrvsshDL+v3I2ta6664bYt27dw+xnXfeOcT++Mc/NritamAGAQAABBQIAAAgoEAAAAABBQIAAAha\nTJPiMcccE2K33357iBU2lFSygbA+ldwHDYloyL333htiX3zxRYgNHTq0qPWyGqKymhSBpujggw8O\nsWKumpgV69OnT1gn6/fjyCOPDLGsJt6sK5I2JcwgAACAgAIBAAAEFAgAACCgQAAAAEGzbFL8z3/+\nE2L33HNPiM2aNaus7aBysYoAAAXcSURBVHfs2DHELr744hCbOHFiiGVdia6SCq+oJUnnnntuiLVt\n27aqeaA2xo8fH2KFYz9rrGY1YO27774h9tZbb4VYVkNX1pXozjrrrBADKiGr8fbMM88MsQkTJoRY\nsVe3zWoYHDFiRN5yNW6p3JQxgwAAAAIKBAAAEFAgAACAgAIBAAAEzbJJ8dprrw2xL7/8sqxt/eQn\nPwmx8847L8R22GGHorb3zTffhNj1118fYgsWLChqe4WyGtBWWGGFEONKd83fpEmTQiyrMavwFuB9\n+/YN62Q11B544IEhNnv27BBbb731Quzss88OsV69euUtDxw4MKwDFMpqQCwc01mNslljdZ999gmx\nrAb2rCbFAQMGhFhra0osxAwCAAAIKBAAAEBAgQAAAAIKBAAAEDT5JsVx48aF2KhRoyq2/WOPPTbE\nim1IzHLllVeG2AMPPBBiU6dOLXsfhbKuLInm75BDDgmx0aNHh9iqq66at3zppZeGdXr06BFinTt3\nDrE5c+aEWNbtbLOaty644IK85awr02U1gqFlmjZtWohlNbcWNiRKsQExa9xkXVU2a6xmNSRm2W67\n7YparzVhBgEAAAQUCAAAIKBAAAAAQZPvQXj33XdD7OOPPy57e5tuumne8m677VbU87LuDDls2LAQ\nO+WUU0Ks2HNgxci6M9n2229fse2jNrLO177wwgshlnURpOeee65ieWT1DWTZbLPNQqzwgjSFvRFS\nvJhSfdtC85J1l9Gs19asi39lXYjruuuuy1su9oJFWXlkvf4We6Gk1o4ZBAAAEFAgAACAgAIBAAAE\nFAgAACBo8k2KxTaYFOv999/PW866cEfWhY0WLlwYYp9++mlRuVWySXHKlCkh1qVLl4ptH7WRdbGY\nrHHTnO4uN2LEiBB7++23Q4wmxeYvq8EvqyGxsPmwvudmXcSrGEOHDg2xrMburLuilrvPlowZBAAA\nEFAgAACAgAIBAAAEFAgAACBo8k2KlVZ4RcSrr766Rpnka9euXYgNHjw4xLp169YI2aCxZTVIZcWu\nv/76EFtjjTXylmt1Rbh77703b3mfffYJ62Q1Xg4cOLBqOaFxTJgwIcSyftbFjvNiFI43Sbr//vuL\nyqM5NfvWEjMIAAAgoEAAAAABBQIAAAgoEAAAQNDkmxS33XbbEOvdu3eIZd0WuhYKG8YkqU2bhuuw\no446KsROP/30iuSEpi+rsXDy5MkhlnWL8UMPPTRv+Z133gnrDBo0aAmyi4YMGRJiF198cd5yVnPY\nWWedVdE80DRk3XI8q2Ew6wqGWWOi8PfhyCOPDOuMHDkyxGbPnh1iJ510Uohx9c7iMIMAAAACCgQA\nABBQIAAAgIACAQAABJZ1K8z6bLHFFj5mzJgqplOcwmYoqfgmrMLjLfZWzIWNYJK00UYbhVhWQ0xr\nYWZj3X2LWudRiqYypos1fvz4ENt1113zlrOOZ0luZXvwwQeHWFYj5PTp/397d4+bMBCFURRa+tRp\nYRGwB2/FDWIRiEWwJ/qUJhENEq1TRvANCiAn/PicjifLcjFCV2Y0fB19Xi6Xcc29Tnm8lTXdrcPh\nELPSWjo9XXO328U12+02ZqXv86ZpYtbnv3a+Zk17gwAABIEAAASBAAAEgQAAhIc/SbFkPp9fNINX\nM5lMYrZerzu7f2kTZOlEvMViEbPT00D7vBGMstFoFLPSqYabzeboc13Xcc1qtYrZbDaLmXV4O28Q\nAIAgEACAIBAAgCAQAIDwlJsUgR/T6bSze5U2Qe73+87uD7cYj8cxK52aWFXVfzxOb3iDAAAEgQAA\nBIEAAAR7EAB4aKeHcJ2b0S1vEACAIBAAgCAQAIAgEACAIBAAgCAQAIAgEACAIBAAgCAQAIAgEACA\nIBAAgCAQAIAgEACAIBAAgDBs2/byi4fDz8Fg8PF3j8OTe2/b9u3eD3ENa5pfWNO8movX9FWBAAD0\ng58YAIAgEACAIBAAgCAQAIAgEACAIBAAgCAQAIAgEACAIBAAgPANTR5np4Tzz/oAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2080eda0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Test the network when training is done\n",
    "x_test, y_test = load_data(mode='test')\n",
    "feed_dict_test = {x: x_test, y: y_test}\n",
    "loss_test, acc_test = sess.run([loss, accuracy], feed_dict=feed_dict_test)\n",
    "print('---------------------------------------------------------')\n",
    "print(\"Test loss: {0:.2f}, test accuracy: {1:.01%}\".format(loss_test, acc_test))\n",
    "print('---------------------------------------------------------')\n",
    "\n",
    "# Plot some of the correct and misclassified examples\n",
    "cls_pred = sess.run(cls_prediction, feed_dict=feed_dict_test)\n",
    "cls_true = np.argmax(y_test, axis=1)\n",
    "plot_images(x_test, cls_true, cls_pred, title='Correct Examples')\n",
    "plot_example_errors(x_test, cls_true, cls_pred, title='Misclassified Examples')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we are finished the testing, we will close the session to free the memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# close the session after you are done with testing\n",
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this step our coding is done. We can inspect more in our network using the __Tensorboard__ open your terminal and type:\n",
    "```bash\n",
    "tensorboard --logdir=logs --host localhost\n",
    "```\n",
    "__NOTE:__ Don't forget to activate your environment !!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open the generated link in your browser."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thanks for reading! If you have any question or doubt, feel free to leave a comment in our [website](http://easy-tensorflow.com/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
